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Executive  Summary 


The  evolutionary  acquisition  strategy  has  been  promulgated  by  the  forthcoming  DoD  In¬ 
struction  5000.2.  It  introduces  innovations  throughout  the  acquisition  cycle:  before  a  con¬ 
tract  is  considered,  technology  readiness  guides  the  choice  of  experiments;  contracts  are  let 
for  one  or  more  blocks;  and  progress  within  each  block  is  managed  with  spiral  develop¬ 
ment.  There  is  some  confusion  as  to  the  nature  of  evolutionary  acquisition  and  spiral  devel¬ 
opment  and  their  relationship.  To  address  these  problems,  a  workshop  was  held  September 
13-15, 2000,  under  joint  sponsorship  of  the  Deputy  Under  Secretary  of  Defense  for  Science 
and  Technology,  the  Software  Engineering  Institute,  and  the  Center  for  Software  Engineer¬ 
ing.  This  report  summarizes  the  workshop  and  presents  its  recommendations. 

The  workshop  concentrated  on  the  application  of  spiral  development  within  the  context  of 
evolutionary  acquisition  by  asking  the  question  **How  can  federal  organizations  use  both  to 
improve  the  overall  acquisition  process?”  Sessions  were  organized  around  the  following  key 
issues: 

•  canonical  definition  of  evolutionary  acquisition 

•  successful  strategies  for  evolutionary  acquisition 

•  the  mapping  between  spiral  development  and  evolutionary  acquisition 

•  institutional  barriers  to  evolutionary  acquisition  and  spiral  development 

•  practical  steps  necessary  to  operationalize  spiral  development  and  evolutionary  acquisi¬ 
tion. 

Presentations 

The  first  day  and  a  half  of  the  workshop  were  devoted  to  presentations  by  executives  and 
practitioners  representing  government,  commercial  users,  solution  providers,  and  contractors. 
In  retrospect,  these  presentations  evoked  these  themes  and  comparisons: 

•  The  definitions  of  EA  and  SD  are  beginning  to  be  understood.  The  basic  definitions 
were  presented  in  detail. 

•  Extensions  of  the  definitions  toward  synchronization,  rapid  deployment,  and  a  new  role 
for  testing. 

•  DoD  funding  and  contracting  policies  are  not  SD  friendly.  A  number  of  speakers  men¬ 
tioned  this  problem,  but  few  documented  it.  Two  offered  partial  solutions. 

•  Stakeholder  teamwork  is  essential.  Speakers  cited  many  situations  where  it  was  in¬ 
valuable. 

•  Education  and  training  are  essential  to  acculturation.  Cultural  change  is  necessary  so 
that  new  approaches  are  not  abandoned  early. 
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VII 


Work  Group  Recommendations 

The  second  half  of  the  workshop  was  devoted  to  small  work  group  sessions,  each  addressing 
a  different  topic.  These  groups  were  charged  with  reconunending  concrete  actions  for  pro¬ 
gress.  They  made  32  recommendations,  falling  into  six  categories: 


Improve  Contract 
Models 

Revise  Funding 
Approaches 

Adapt  Acquisition 
Policies 

Enhance  Integrated 
Product  Teams 

Provide  Training 


Study  SD  and  EA 


Incorporation  of  EA  and  SD  in  projects  requires  new 
contract  clauses,  new  kinds  of  contracts,  and  new 
procedures  for  letting  contracts. 

Innovative  funding  approaches  are  necessary  to  adapt  to 
SD  and  EA. 

Acquisition  policies  must  be  adapted  to  suit  EA  and  SD. 


IPTs  must  be  given  authority  and  status  to  match  their 
responsibility. 

People  need  new  skills  to  deal  with  EA  and  SD.  These 
skills  can  come  from  post-secondary  education,  service¬ 
wide  training,  and  training  within  specific  projects. 

SD  and  EA  require  further  study  in  order  to  validate  and 
improve  the  processes. 


For  each  recommendation,  the  work  group  proposed  an  action  agent,  the  person  or  group 
most  appropriate  for  taking  the  necessary  actions  and  a  date  by  which  that  action  ought  to 
have  occurred. 
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Abstract 


The  evolutionary  acquisition  strategy  has  been  promulgated  by  the  forthcoming  DoD  Instruc¬ 
tion  5000.2.  It  introduces  innovations  throughout  the  acquisition  cycle:  before  a  contract  is 
considered,  technology  readiness  guides  the  choice  of  experiments;  contracts  are  let  for  one 
or  more  blocks;  and  progress  within  each  block  is  managed  with  spiral  development.  There  is 
some  confusion  as  to  the  nature  of  evolutionary  acquisition  and  spiral  development  and  their 
relationship.  To  address  these  problems,  a  workshop  was  held  September  13-15,  under  joint 
sponsorship  of  the  Deputy  Under  Secretary  of  Defense  for  Science  and  Technology,  the  Soft¬ 
ware  Engineering  Institute,  and  the  Center  for  Software  Engineering.  This  report  summarizes 
the  workshop  and  presents  its  recommendations. 

Themes  appearing  in  the  workshop  presentations  included  the  lack  of  understanding  of  the 
definitions  of  evolutionary  acquisition  and  spiral  development,  some  extensions  to  these  defi¬ 
nitions,  the  barriers  imposed  by  existing  funding  and  contracting  policies,  the  need  for  team¬ 
work  among  all  stakeholders,  and  the  role  of  education  and  training  in  acculturation.  Work 
groups  at  the  workshop  recommended  specific  actions  aimed  at  building  and  spreading  a  cul¬ 
ture  for  evolutionary  acquisition  and  spiral  development.  These  actions  can  be  grouped  under 
the  topics  of  improvements  to  contract  models,  revision  of  funding  approaches,  adaptation  of 
acquisition  policies,  enhancement  of  integrated  product  teams,  training  and  acculturation  of 
participants,  and  studies  of  evolutionary  acquisition  and  spiral  development  to  validate  and 
improve  them. 
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ix 


1  Introduction 


The  recent  DoD  revisions  to  the  5000.2  series  have  introduced  evolutionary  acquisition  (EA) 
as  the  appropriate  method  of  operation  for  acquiring  new  and  enhanced  systems.  The  ap¬ 
proach  begins  with  a  series  of  explorations  to  ensure  technology  readiness  and  then  proceeds 
to  doing  spiral  development  (SD)  in  each  of  a  number  of  funding  blocks.  To  study  and  popu¬ 
larize  the  new  policies,  a  workshop  was  conducted  September  13-15,  2000,  under  the  aegis  of 
Dr.  Delores  Etter — the  Deputy  Under  Secretary  of  Defense  for  Science  and  Technology 
(DUSD/S&T) — and  the  sponsorship  of  the  Software  Engineering  Institute  (SEI)  of  Carnegie 
Mellon  University  and  the  Center  for  Software  Engineering  (CSE)  of  the  University  of 
Southern  California.  This  report  is  the  result. 

Dr.  Etter  opened  the  workshop  with  a  keynote  address  describing  the  challenges  to  DoD  sys¬ 
tems  and  software  development.  Later  presentations  included  one  from  Dr.  Barry  Boehm 
defining  the  SD  and  another  from  Dr.  Jack  Ferguson  describing  the  new  DoD  5000.2  policies. 
The  workshop  was  organized  around  five  topics,  as  listed  in  Table  1.  This  table  also  lists  the 
presenters  who  spoke  to  each  topic  on  the  first  day.  The  second  day  was  devoted  to  five  work 
groups,  each  addressing  one  of  the  topics.  The  third  morning  featured  presentations  from  the 
five  groups  and  a  summary  by  Steve  Cross,  Director  of  the  SEI. 

Section  3  presents  an  overview  of  the  workshop  themes. 

•  Indented  blocks  in  this  font  are  from  the  slide  presentations 
The  fourth  section  summarizes  the  recommendations  from  the  work  groups  and  the  next  sec¬ 
tion  presents  their  findings.  A  final  section  compares  this  workshop  with  a  prior  workshop  on 
spiral  development  held  in  February  2000.  An  appendix  provides  a  sorted  list  of  the  recom¬ 
mendations.  Supplementary  material,  including  the  presentations,  can  be  found  on  the  work¬ 
shop  Web  site; 

http://www.sei.cmu.edu/cbs/spiral2000 

The  Workshop  met  in  the  large  conference  hall  on  the  first  floor  of  the  Arlington,  Virginia, 
building  that  houses  the  SEI.  The  five  dozen  attendees  were  seated  at  five  rows  of  six  tables 
with  a  central  aisle.  Time  was  managed  with  a  PowerPoint  application  showing  the  minutes 
remaining;  when  time  expired,  it  sounded  a  bell. 

Presenters  covered  both  the  government  and  the  contractor  sides  of  acquisition,  with  a  few 
speakers  from  the  commercial  sector  to  give  external  viewpoints.  There  were  no  representa¬ 
tives  from  organizations  to  which  acquired  systems  would  be  deployed.  There  were  numer¬ 
ous  references  to  systems  developed  with  SD,  but  only  one — ^the  Joint  Expeditionary  Forces 
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Experiment  (JEFX)  as  described  by  Tillotson — was  described  in  enough  detail  to  make  dis¬ 
cernible  its  spiral  development  aspects. 

Table  1:  Presentations  at  the  Spiral  Workshop _ 

Keynote  speaker 

Dr.  Delores  M.  Etter,  DUSD  (S&T)  -  Spiral  Workshop _ 

“Definition”  -  Canonical  DeHnition  of  EA 

Barry  Boehm,  USC  -  Spiral  Development  &  Evolutionary  Acquisition:  Where  Are  We 
Today? 

Jeff  Rothenberg,  Rand  -  Spiral  Development  &  Evolutionary  Acquisition 
_ Jack  Ferguson,  OSD  -  Spiral  Workshop _ 

“Mapping”  -  The  Mapping  between  SD  and  EA 

Maj.  Ross  McNutt,  USAF  -  Reducing  Air  Force  Acquisition  Response  Times:  Evolu¬ 
tionary  Acquisition  and  Spiral  Development 

_ Eliot  Axelband,  USC  -  Comments/Observations/Suggestions _ _ 

“Strategies”  -  Successful  Strategies  for  EA 

_ Charles  Leinbach,  C-Bridge  -  c-Commerce  and  Spiral  Development _ 

^‘Barriers”  -  Institutional  Barriers  to  EA  and  SD 

Stan  Levine,  US  Army  -  Implementing  System  of  Systems  in  a  Spiral  Develop¬ 
ment/Evolutionary  Acquisition  Environment 

_ Don  Andres,  TRW  -  Evolutionary  Acquisition  &  Spiral  Development:  Barriers _ 

“Operationalization”  -  Practical  Steps  Necessary  to  Operationalize  SD  and  EA 
Michael  Bloom,  MITRE  -  Spiral  Development  in  DoD:  Get  out  of  the  box! 

Tony  Jordano,  SAIC  -  SAIC  -  Spiral  Development 

Larry  McKee,  MITRE  -  Spiral  Development:  A  User’s  Perspective 

Col.  Dave  Tillotson,  USAF  -  Joint  Expeditionary  Forces  Experiment  (JEFX) _ 
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2  Keynote  Speaker:  Dr.  Delores  Etter 


In  her  keynote.  Dr.  Delores  Etter  described  DoD  acquisitions  in  Research,  Development, 
Testing  and  Evaluation  (RDT&E),  the  efforts  to  ensure  a  maximum  national  secunty  payoff 
from  these  acquisitions  and,  more  broadly,  the  methods  used  to  acquire  systems  throughout 
the  armed  forces.  As  Figure  1  shows,  over  40  billion  dollars  will  be  invested  this  ye^in 
RDT&E,  with  a  bit  over  20%  going  to  Dr.  fitter’s  area  of  Science  and  Technology  (S&T). 


Operating  $15.6B 


Development  $16.7B 


Science  and 
Technology  $9.0B 


Operational  Systems 
Development  ($13.0B) 


RDT&E  Management 
Support  ($2.6B) 

Engineering  and 
Manufacturing 
Development  ($8.8B) 

Demonstration  and 
Validation  ($7.9B) 

Advanced  Technology 
Development  ($4.0B) 

Applied  Research  ($3.7B) 
Basic  Research  ($1 .3B) 


Figure  1:  Fiscal  Year  2001  Spending  on  RDT&E,  $41.3  Billion 
In  fiscal  year  2001,  spending  on  S&T  is  about  equally  distributed  among  the  services  and 

Defense  Advances  Research  Projects  Agency  (DARPA).  Despite  an  era  of  decreasing  de- 

fense  budgets,  S&T  spending  by  the  services  has  remained  more  or  less  steady  over  the  last 
decade,  as  shown  in  Figure  2. 
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Figure  2:  Annual  Spending  by  the  Services  on  Science  and  Technology 

Although  Air  Force  spending  has  declined  in  S&T,  its  total  spending  on  RDT&E  remains  far 
above  that  of  the  other  services  on  a  per-warrior  basis.  Figure  3  (not  from  the  presentation) 
shows  the  total  obligation  authority  of  the  three  services  as  a  number  of  dollars  per  person- 
nel-dollar.  The  RDT&E  spending  is  18,  32,  and  65  cents  per  personnel  dollar  for  the  Army, 
Navy,  and  Air  Force,  respectively.  These  factors  reflect  the  facts  that  the  Navy  and  Air  Force 
are  more  asset-intensive  than  the  Army,  and  the  Air  Force  has  a  lower  ratio  of  officers  to  en¬ 
listed  personnel  (AF:1.1,  Navy:  2.6,  Army:  1.6). 

O&M 


Air  Force  ■  Navy  ■  Army 

National  Defense  Budget  Btimates  FY01 


Figure  3:  FY01  Spending  ($  per  Personnel  $) 

Dr.  Etter  views  DoD  Science  and  Technology  as  a  partnership  among  DARPA,  the  universi¬ 
ties,  the  service  laboratories,  industries,  interagency  efforts,  and  international  coalition  coop¬ 
eration.  She  sees  these  groups  as  approaching  software  intensive  systems  fi-om  five  direc¬ 
tions.  In  the  following  list,  the  examples  are  chosen  from  her  presentation: 
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•  Policy:  Acquisition  Reform  Legislation;  AT&L  5000  Rewrite 

.  Discipline:  Capability  Maturity  Model®  Integration  (CMMI™);  Improving  DoD 

Software  Metrics 


Collaboration:  Office  of  the  Under  Secretary  of  Defense  of  Acquisition, 
Technology  and  Logistics  OUSD(AT&L)  Tri-Service  Assessment  Initiative 


Career  Development:  Developing  a  training  course  for  S&T  managers  in  the 
acquisition  of  software-intensive  systems 

Science  and  Technology:  Building  and  strengthening  relationships  with 
researchers  in  software  engineering  technology  and  education 


Turning  to  evolutionary  acquisition  and  the  workshop.  Dr.  Etter  pointed  out  that  “Getting 
evolutionary  acquisition  to  succeed  for  DoD  software-intensive  systems  requires  new  direc¬ 
tions  and  harmonizing”  these  areas. 

•  software  acquisition  practices 

•  milestone  definitions 

•  maturity  models 

•  metrics 

•  product  line  management 

•  process  and  architecture  technology 

•  education  and  training 


Considering  all  these  aspects,  she  concluded  that: 

Evolutionary  Acquisition  is  the  way  DoD  must  go 
to  provide  needed  capability  to  the  warfighter 
in  a  timely  manner 


®  Capability  Maturity  Model  and  CMM  are  registered  in  the  U.S  Patent  and  Trademanrk  Office, 
CMMI  is  a  service  mark  of  Carnegie  Mellon  University  _ _ 
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3  Themes 


A  number  of  topics  appeared  in  multiple  presentations,  work  group  recommendations,  and 
comments  after  the  workshop.  This  section  collects  together  these  themes. 

The  definitions  of  EA  and  SD  are  beginning  to  be  understood. 

The  definition  of  SD  should  be  extended. 

DoD  funding  and  contracting  policies  are  not  SD  friendly. 

Stakeholder  teamwork  is  essential. 

Education  and  training  are  essential  to  acculturation. 

Each  theme  is  discussed  in  a  following  subsection. 

3.1  The  Definitions  of  EA  and  SD  Are  Beginning  to  Be 
Understood 

A  fundamental  basis  of  a  workshop  on  spiral  development  and  evolutionary  acquisition  ought 
to  be  a  firm  understanding  of  the  definitions  of  each  of  these  terms.  The  workshop  did  indeed 

shed  light  on  them. 

Evolutionary  Acquisition:  The  DoD  5000  series— DoD  Directive  5000.1,  DoD  Instruction 
5000.2,  and  DoD  Regulation  5000.2R— was  rewritten  under  Dr.  Etter  in  the  office  of  Soft¬ 
ware  Intensive  Systems  under  Jack  Ferguson.  On  the  second  morning  of  the  workshop  Fer¬ 
guson  discussed  these  documents  and  their  new  “Evolutionary  Acquisition”  approach.  He 
began  with  the  objectives  of  the  rewrite: 

•  Develop  a  new  acquisition  model  that  reduces  cost  and  cycle  time  while  delivering 
improved  performance. 

•  Move  DoD  closer  to  a  commercial-style  approach. 

•  Implement  Section  912  recommendations. 

•  Implement  other  reports  and  key  initiatives. 

•  Further  streamline  the  acquisition  process. 

Here  the  “Section  912  study”  refers  to  a  study  mandated  by  Congress  to  exanune  acquisition. 
It  had  a  major  focus  area  of  “cycle  time,”  the  length  of  time  taken  to  perform  an  acquisition. 
The  fact  that  this  is  a  problem  is  evident  in  that  both  Ferguson  and  McNutt  presented  Figure  4 
showing  that  the  cycle  time  from  program  initiation  until  “IOC”  is  approaching  a  decade.  It 
must  be  understood  here  that  IOC  is  not  the  completion  of  the  project,  but  is  instead  the  Ini¬ 
tial  Operational  Capability,  that  is,  the  first  time  anything  is  made  available  to  the  troops  in 
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the  field.  It  is  suggestive  that  the  curves  are  an  inverse  of  the  spending  curves  in  Figure  2,  but 
neither  presenter  commented  on  this. 


Figure  4:  Average  Time  from  Program  Start  to  IOC 


Ferguson  emphasized  these  aspects  of  the  912  study’s  focus  on  acquisition  cycle  time: 

•  Time-Based  Requirements 

•  Delay  Freezing  of  Requirements 

•  Requirements  Set  at  Milestone  II  not  Milestone  I 

•  Evolutionary  Acquisition  as  Norm 

Turning  to  evolutionary  acquisition  itself,  Ferguson  showed  the  definitions  from  two  Air 
Force  documents: 

Evolutionary  Acquisition  (EA)- An  acquisition  strategy  to  adapt  to  a  changing  environment 
by  rapidly  acquiring  and  sustaining  a  supportable  core  capability  and  incre¬ 
mentally  inserting  new  technology  or  additional  capability  [SAF/AQ  00]- 

Evolutionary  Acquisition  (EA)  -  Acn  acquisition  strategy  whereby  a  basic  capability  is  fielded 
with  the  intent  to  develop  and  field  additional  capabilities  as  requirements 
are  refinedXSAF! AQi  00]- 

In  DoDI  5000.2  [USD(AT«&L)  01],  evolutionary  acquisition  is  defined  thus: 

In  an  evolutionary  approach,  the  ultimate  capability  delivered  to  the  user  is  di¬ 
vided  into  two  or  more  blocks,  vyith  increasing  increments  of  capability.  Deliver¬ 
ies  for  each  block  may  extend  over  months  or  years.  Block  1  provides  the  initial 
deployment  capability  (a  usable  increment  of  capability  called  for  in  the  ORD). 

There  are  two  approaches  to  treatment  of  subsequent  blocks: 

The  ORD  includes  a  firm  definition  of  full  capability,  as  well  as  a  firm  definition 
of  requirements  to  be  satisfied  by  each  block  . . . 
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or 

The  ORD  includes  a  firm  definition  of  the  first  block,  but  does  not  allocate  to 
specific  subsequent  blocks  the  remaining  requirements  that  must  be  met  to 
achieve  full  capability.  . . . 

The  full  scope  of  the  5000  acquisition  process  extends  from  initial  notion  all  the  way  to  mul¬ 
tiple  successive  fieldings  of  the  program.  The  concept  of  technology  readiness,  as  defined  by 
a  Government  Accounting  Office  report  [GAO  99],  is  utilized  to  test  technologies  to  prepare 
them  for  actual  acquisition.  As  shown  in  Figure  5  the  technology  is  hatched  somewhere  and 
adopted  as  an  S&T  project  to  try  it  out.  If  successful  there,  it  becomes  the  subject  of  ATD, 
ACTD  (Advanced  [Concept]  Technology  Demonstration),  or  JWE  (Joint  Warfare  Exercise). 
Somewhere  in  this  phase  a  decision  is  reached  to  actually  acquire  a  system  embodying  the 
technology. 


S&T  Demonstration  Acquisition  Program 

Projects  Projects 


Figure  5:  DoDI  5000.2  Acquisition  Process:  from  Idea  to  Fielding 


It  is  at  the  Acquisition  Program  phase  that  evolutionary  acquisition  commences.  The  key 
concept  is  that  the  acquisition  occurs  in  a  sequence  of  blocks,  with  each  block  culminating  in 
fielding  some  fraction  of  the  program’s  capability.  That  capability  is  sketched  in  the  contract 
for  the  block,  but  is  probably  not  spelled  out  completely  in  order  to  give  scope  for  the  deci¬ 
sions  of  the  spiral  development  risk-management  team.  Indeed,  Ferguson  noted  that 

•  Evolutionary  Acquisition  implies  evolutionary  requirements 

Directed  in  CJCS  Instruction  31 70.01  A  Requirements  Generation  System 

•  Evolutionary  Acquisition 

also  implies  evolutionary  fielding 
with  impacts  on  training  and  sustainment 

Within  blocks  of  evolutionary  acquisition,  DoDI  5000.2  specifies  an  “iterative  spiral  ap¬ 
proach”  be  used  for  development. 

Spiral  development  was  fully  defined  in  Boehm’s  talk,  a  refinement  of  his  talk  at  the  Febru¬ 
ary  workshop.  (His  talk  also  described  the  results  of  the  first  workshop.)  Boehm  defined  spi¬ 
ral  development  as 
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•  A  risk-driven  process  model  generator 

•  Used  to  guide  concurrent  engineering 

•  Two  distinguishing  features; 

-  cyclic  approach  for  growing  system  definition 

-  anchor  point  stakeholder-commitment  milestones 

He  went  on  to  specify  six  attributes  that  are  “invariant”  in  the  sense  that  every  spiral  process 
ought  to  have  them.  In  an  extension  of  the  February  presentation,  he  showed  how  IEEE 
12207.2-1997  [lEEE/EIA  98]  helps  to  choose  among  the  various  acquisition  models.  This 
choice  is  made  in  each  cycle  of  a  spiral  development,  based  on  the  risk  factors  deemed  of 
highest  priority  at  the  start  of  that  cycle. 

Despite  all  these  efforts  at  defining  the  process  models,  there  was  still  some  confusion.  After 
the  conference,  attendee  Peter  Hantos  from  Xerox  remarked  that 

I  think  many  people  were  not  clear  on  the  connection  between  EA  and  SD. . . . 

Getting  straight  the  basic  terminology  was  also  a  common  [question].  (What  is  a 
software-intensive  system,  what  is  the  reference  definition  of  spiral  development, 
why  is  the  acquisition  “evolutionary,”  and  how  does  it  relate  to  the  evolutionary 
spiral  development,  etc.)  . . .  Surprisingly  there  was  a  lot  of  debate  on  very  basic 
definitions. 

3.2  Extensions  to  the  Definition  of  SD 

A  number  of  speakers  echoed  or  extended  the  definitions  of  spiral  development.  McNutt  reit¬ 
erated  Ferguson’s  definition,  emphasizing  the  role  of  spiral  development  in  each  iteration.  In 
a  further  refinement,  McKee  described  a  “CAOC-X”  unit  in  the  Air  Force  that  will  serve  as  a 
testing  area  and  will  help  manage  the  spiral  development  of  each  project.  From  a  contractor 
perspective,  Jordano  showed  how  spiral  development  is  institutionalized  by  the  corporate 
process  group.  There  is  a  definition  of  spiral  development  and  a  catalog  of  criteria  to  use  to 
determine  if  spiral  development  is  correct  for  any  particular  project. 

Two  speakers  emphasized  that  spiral  project  development  does  not  occur  in  a  vacuum;  there 
is  a  persistent  need  to  synchronize  the  activities  in  the  spiral  development  with  those  in  other 
development  projects.  Rothenberg  phrased  the  problem  this  way. 

•  We  split  complex  endeavors  into  simpler  activities  to  “divide  and  conquer”  them. 

•  But  the  resulting  activities  are  not  motivated  or  funded  to  look  beyond  their  own 
boundaries. 

•  So  they  ignore  external  changes  that  undermine  their  assumptions  and  decisions. 

He  illustrated  the  problem  with  the  example  of  the  Abrams  tank  project.  Figure  6.  The  Pro¬ 
gram  Manager  (PM)  for  this  project  must  deal  with  concerns  from  (at  least)  three  other  PMs, 
four  Program  Element  Officers  (PEOs),  and  three  multinational  arenas. 
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Figure  6:  External  Programs  with  which  the  Abrams  Tank  Must  Synchronize 


Rothenberg  defined  an  enhanced  spiral  cycle  to  deal  with  this  sort  of  problem.  To  over- 
simplify,  each  spiral  cycle  has  five  steps  (where  asterisks  indicate  additions  to  a  normal  spiral 

process): 

•  Define  alternatives,  ‘consider  external  factors. 

•  Evaluate  alternatives,  based  on  risk;  choose  one. 

•  Perform  this  cycle’s  work. 

•  Evaluate  and  ‘post  results;  plan  next  cycle,  ‘synchronize,  and  commit. 

•  ‘(Re-)  Identify  external  factors  and  stakeholders;  negotiate  reconciliations. 

In  a  similar  vein,  Levine  noted  that 

.  The  Army  must  synchronize  “System  of  Systems”  mission  threads  and  interoperability 
requirements  across  the  Army  and  then  on  to  Joint  and  Coalition  levels. 

In  elaboration,  he  presented  material  in  each  of  these  areas: 

•  Requirements  Synchronization 

•  System  Design  Synchronization 

•  System  Implementation  Synchronization 

•  Operational  Integration  Activities 

Sometimes  the  need  for  synchronization  is  expressed  as  “concurrent  engineering.”  Boehm 
mentioned  that  spiral  development  is  “used  to  guide  concurrent  engineering,”  and  Bloom 
noted  that  a  “Concurrent  Engineering  Approach”  is  essential  to 

•  Understand  the  problem  and  try  potential  solutions 

This  approach  must  be  applied  in  four  areas:  system  context,  architecture/design,  legacy  sys¬ 
tem,  and  marketplace. 
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Although  not  strictly  a  part  of  the  definition  of  evolutionary  acquisition,  several  speakers 
talked  about  the  anticipated  role  of  that  approach  and  spiral  development  in  reducing  the  time 
to  complete  projects.  The  approach  is  actually  realistic  to  the  extent  that  it  does  not  mean  get¬ 
ting  the  project  “done”  sooner,  but  rather  the  expectation  is  that  partial  results  can  be  de¬ 
ployed  to  the  field  earlier  in  the  development  process.  McNutt’s  entire  presentation  was  on 
the  ongoing  effort  to  reduce  the  “cycle  time”  to  field  a  project.  As  part  of  this  he  presented  a 
method  for  analyzing  the  “cost  of  delay,”  which  he  showed  to  far  outweigh  other  considera¬ 
tions  in  some  cases.  He  expects  evolutionary  acquisition  and  spiral  development  to  play  a  key 
role  in  cycle  time  reduction.  Levine  agreed  and  stated  that  combining  the  two 

•  Accelerates  acquisition  and  provides  recurring  improvements  to  systems  fielded  to  our 
warfighters 

Leinbach  observed  the  same  need  for  faster  delivery  in  the  commercial  sector  where  the  cus¬ 
tomers  “need  fast  business  deployment,”  and  get  it  through  his  company’s  “RAPID™”  proc¬ 
ess,  which  provides 

•  Application  delivered  much  sooner  at  less  cost 

•  First  release  available  very  quickly 

In  other  words,  evolutionary  acquisition  fields  an  early  version  of  the  system,  which  is  opera¬ 
tional,  but  probably  without  full  capabilities.  Typically,  it  is  not  expected  in  DoD  5000  that 
each  spiral  will  field  its  results,  although  that  is  common  in  spiral  development  projects. 

The  role  of  testing  was  another  area  where  spiral  development  was  extended.  In  DoD  acqui¬ 
sition,  testing  has  come  to  be  done  by  an  independent  organization  operating  after  the  system 
is  delivered.  This  makes  sense  for  large  military  hardware  systems,  where  the  military  alone 
is  equipped  to  assess  the  battlefield  capabilities  of  a  tool.  For  software,  however,  test-at-the- 
end  tends  to  delay  rather  than  inform  the  acquisition  process.  Tillotson  noted  this  in  his  sum¬ 
mary  where  he  remarked 

•  Assessment  or  test  must  be  part  of  initial  planning 

In  remarks  written  after  the  workshop.  Bloom  said: 

Testing  is  “broken"  for  C4I  (Command,  Control,  Computers,Communications 
and  Intelligence)  systems.  Application  of  preproduction  testing  techniques 
designed  for  an  “iron"  world  are  barriers  to  timely  fielding  of  software  updates. 

A  risk-based— not  risk-averse — approach  needs  to  be  developed  and  employed 
similar  to  that  used  by  commercial  organizations  like  Microsoft. 

Bloom  devoted  half  of  his  presentation  to  the  need  for  and  possibilities  of  testing  as  part  of 
spiral  development.  He  created  a  parable-picture  with  the  task  of  moving  cows  from  one  hill¬ 
top  field  to  another.  The  legacy  system  is  a  trail  through  the  valley  and  the  modem  system  is 
a  bridge.  The  role  of  testing  in  spiral  development,  he  says,  is  to  test  those  portions  of  the 
system  that  are  released  in  each  iteration.  There  is  no  need  to  gamble  by  switching  everything 
to  the  new  system  immediately.  For  the  cows,  we  can  do  tests  of  the  initial  release  by  sending 
only  one  across  the  bridge,  as  in  Figure  7. 
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Figure  7:  Testing  the  New  System  with  Only  a  Portion  of  the  Load 


Bloom’s  point  was  that  it  is  time  to  combine  testing  considerations  with  risk  management. 

He  said  that  testing  should  be 

•  Continuous  rather  than  end  of  block 

•  Subject  to  a  risk  assessment  (vs.  benefit  of  testing)  for  the  type  of  product  and  its 
intended  environment 

•  An  integral  part  of  experimentation  to  capture  the  utility  and  underlying  need  for 
successful  prototypes 

•  Used  to  establish  an  expanding  operational  “flight  envelope”  as  a  new  capability 
matures  in  terms  of  functionality,  reliability,  usability,  efficiency,  maintainability, 
interoperability,  security... 
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•  Funding  constraints  (Fiscal  Year,  PPBS,  POM  process,  Colors  and 
Hues  of  Money,  etc.) 

•  Changing  DoD  personnel  -  every  three  years 

•  Risk  adverse  operating  philosophy 

•  Becoming  too-Process-focused  Vs.  Product-focused 

•  Cumbersome  DoD  reporting  mechanisms 

•  Need  to  establish  how  Spiral  works  contractually  in  a  fixed  price 
environment 

•  Industry  will  not  accept  fixed  price  spiral  development 

•  Restructure  and  strengthen  contract  incentives 

•  EA  and  SD  is  not  a  good  match  for  typical  fixed  price  contracting 
Evolutionary  acquisition  needs 

•  Flexible  funding  process 
Challenges  include 

•  Laws,  Policy,  and  Rules 
Project  approval  time  is  lengthy: 

•  24  separate  reviews  within  the  Air  Force  Pentagon  alone 

-  process  takes  two  to  five  years  to  complete 

Budgeting,  financing,  contracting,  and  other  support  systems  need  to  be 
further  reformed.  . . .  EA  requires  direction  and  harmonizing  of 
acquisition  practices  (policy,  collaboration,  career  development,  and 
discipline). 


Despite  this  almost  universal  agreement  that  problems  exist,  there  were  few  suggestions  at 
the  workshop  of  steps  that  need  to  be  taken  or  are  being  taken.  Two  speakers  did  present  the 
following. 


Ferguson  explicitly  noted  that 

•  Evolutionary  Acquisition  implies  evolutionary  requirements 

Directed  in  CJCS-I  3170.01  A  Requirements  Generation  System 

In  other  words,  contracts  can  and  must  be  written  without  complete  specification  of  all  re¬ 
quirements.  It  seems  then,  although  Ferguson  did  not  say  so,  that  no  project  contracted  for 
under  these  provisions  can  ever  “fail.”  The  contract  specifies  a  general  goal,  a  set  number  of 
dollars,  and  a  set  time.  It  is  the  task  of  the  spiral  development  management  team  to  plan  work 
toward  that  goal  so  that  it  fits  within  the  time  and  resources  available.  In  a  sense,  all  contracts 
become  contracts  for  time  and  materials. 


McNutt  noted  that  one  problem  is  that  funds  are  continuously  stretched  and  reallocated  to 
cover  whatever  projects  seem  important  at  a  time.  This  means  an  uncertainty  in  completion  of 
any  one  project  and  an  accumulation  of  incomplete  projects.  He  recommended  an  alternate 
approach: 
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•  Require  all  projects  that  are  initiated  to  be  fully  funded  based  on  development  related 
requirements. 

•  Establish  an  effective  project  screening  process. 

•  Limit  the  number  of  projects  in  each  phase  of  development. 

•  Clear  the  logjam  of  current  projects. 

•  Ensure  necessary  funds  are  available  to  accelerate  projects  as  opportunities  arise. 

One  way  to  reduce  the  project  initiation  time,  McNutt  suggested,  is  to  overlap  the  planning  of 
a  project  with  the  technology  readiness  preparations  that  precede  letting  of  a  contract,  as 
shown  in  Figure  8. 


Current _ _ 

Experimeri  ATDs  -  '  Analyze 
.  Battlete.#kC!tDs  ;  ^  Options 


< -  2-5  years - > 

Ran  Project: 

Scope,  Funding,  Schedule 


Proposed 


Decision  to  proceed  based 
on  experiment  results 


Figure  8:  Project  Initiation:  Current  Leviathan  and  Proposed  Dolphin 


3.4  Stakeholder  Teamwork  Is  Essential 


Risk  management  is  vital  to  spiral  development,  but  no  single  individual  can  cope.  Risks 
must  be  managed  by  a  team  in  order  to  discover  all  the  ramifications  of  any  decision.  Team¬ 
work  must  extend  outward  from  the  team  doing  risk  management  to  all  other  participants  so 
an  unadulterated  stream  of  information  percolates  upward  and  illuminates  all  risk  considera¬ 
tions.  This  need  for  teamwork  was  emphasized  by  a  number  of  speakers: 


Jordano  in  his  summary  remarked 

•  PMO  And  Contractor  Need  To  Work  Together  As  Partners. 


Levine  said  (emphasis  in  the  original): 

•  The  Army  and  OSD  must  establish  an  acquisition  atmosphere  that  encourages  its 
multiple  PEOs,  PMs,  and  contractors  to  collaborativelv  work  together. 

Tillotson  talked  about  using  a  spiral  approach  to  manage  the  development  of  the  entire  JEFX 
project,  involving  hundreds  of  separate  experimental  systems/development  projects  and  thou¬ 
sands  of  personnel.  The  spiral  approach,  he  said,  had  these  advantages. 

•  Early  user  involvement 

•  Promotes  Teamwork 
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He  added  that  tight  user  involvement  was  the  key  to 

•  Timely  and  focused  system  changes 

•  T ransition  of  results  to  the  field 

Andres  had  the  most  to  say  about  the  need  for  teamwork.  First  of  all,  he  said,  there  is  a  need 
for  a  common  vision  shared  by  all  members  of  the  project; 

•  A  common  visionary  goal  needs  to  be  established  that  is  common  to  the  partnership  - 
Users,  SPO,  Contractor. 

•  The  definition  of  the  end  target  must  be  understood. 

•  Common  expectations,  evolution  plan 

However,  he  observed  a  number  of  impediments  to  such  a  vision  and  shared  teamwork: 

•  Near-term  desires  vs.  long-term  strategic  goals 

•  Changes  in  the  vision  with  no  plan 

•  Multiple  customers/users 

•  Not  putting  the  right  team  in  place  -  contractor 

•  Associate  contractor  agreements  (ASCONs) 

Despite  the  near-universal  agreement  on  the  need  for  teamwork,  no  speaker  had  much  to  say 
about  how  to  attain  this  state.  The  work  groups,  however,  made  a  number  of  recommenda¬ 
tions  in  this  area. 

3.5  Education  and  Training  Are  Essential  to  Accul¬ 
turation 

Given  its  huge  number  of  personnel,  the  DoD  has  an  extensive  training  budget  and  effort.  It 
also  harbors  a  number  of  cultures  and  traditions.  Sometimes  the  two  conflict.  Nonetheless, 
training  is  one  of  the  few  avenues  open  to  DoD  to  affect  traditions  and  culture  and,  in  gen¬ 
eral,  the  behavior  of  its  personnel.  Workshop  attendees  and  presenters  noted  a  number  of 
situations  in  which  traditions  and  culture  are  detrimental  to  achievement  of  the  gains  ex¬ 
pected  from  spiral  development  and  evolutionary  acquisition.  They  also  noted  the  value  of 
training  in  enhancing  those  gains. 

Jordano  summarized  by  demanding  training  of  both  the  acquirers  and  the  “acquirees”: 

•  Acquisition  Agency  and  PMO  Need  To  Be 

-  trained  and  mature 

-  at  Least  Level  2  of  SA-CMM  (Software  Acquisition  CMM) 

•  Contractor  Needs  To  Be 

-  trained  and  mature 

-  level  3  of  SW-CMM  (Software  development  CMM) 

Joe  Ferrara  (SEI)  later  presented  this  approach  to  such  training: 


16 


CMU/SEI-2001-SR-005 


Not  surprisingly,  all  the  work  groups  said  something  about  education  and 
training.  This  is  critically  important.  I  think  SEI/CSE  should  arrange  to  meet 
with  the  Defense  Acquisition  University  (DAU)  to  discuss  this  point.  I  like  the 
idea  of  tailoring  courses  to  different  stakeholders.  Also,  as  part  of  the  education 
process,  I  think  exchange  programs  within  government  and  between  government 
and  industry  can  be  helpjul.  For  example,  given  the  complaints  about  the  budget 
process,  it  would  be  useful  to  have  an  acquisition  manager  spend  a  year  as  a 
budget  examiner  to  experience  the  pressures  and  group  norms.  Similarly,  it 
would  be  enlightening  to  have  a  budget  officer  spend  a  year  in  a  program  office. 

This  could  be  a  longer  term  program  aimed  at  improving  understanding  across 
communities. 

Thomas  Bono  (MITRE)  remarked: 

The  signing  of  policies  such  as  DoD  5000,  AH  63-123,  and  other  regulations  does  not 
institutionalize  them  in  the  SPOs  that  execute  acquisition.  PMs  tend  to  revert  back  to 
what  they  know  how  to  do  best  if  not  properly  made  aware  of  policies  and  given  train¬ 
ing,  step-by-step  instructions,  and  artifact  examples. 


However,  McNutt  felt  that  as  far  as  “institutionalizing  EA  and  SD”  we  have 

•  Made  good  progress,  but  still  have  ways  to  go  to 

-  incorporate  into  acquisition  processes 

-  institutionalize  in  our  processes 

•  Good  progress  with  AF  Acquisition  Community,  Contractors,  OSD(A&T) 

•  Only  to  convince  the  planning,  requirements,  programming,  costing,  MAJCOM, 

training,  sustainment,  test,  finance,  administration,  congress,  gao.cbo.afia, . :-) 


An  important  aspect  of  the  culture  is  the  “rice  bowl”  mentioned  by  McKee: 

•  Culture  and  “Rice  Bowls” 

This  has  reference  to  the  practice  of  senior  leadership  personnel  ensuring  that  funds  are  found 
to  support  their  special  project  groups,  regardless  of  the  actual  need  for  these  groups.  McKee 
expanded  on  this  point  in  remarks  made  after  the  conference. 

Culture  is  perhaps  the  most  difficult  and  time-consuming  piece  of  the  puzzle.  We  need 
documented  case  studies  to  convince  and  tell  the  story.  It  will  not  happen  overnight. 
Significant  “proof’  of  this  can  even  be  found  in  many  of  the  introductory  briefings 
given^a  lot  of  “obstacles  ’’  identified.  While  perhaps  valid,  they  are  in  my  opinion 
mostly  cultural  issues— merely  bumps  in  the  road  if  we  stay  the  course.  As  identified  in 
the  end  -  policy,  regulations,  etc.,  all  permit  SD.  So  why  aren’t  we  doing  it?  Pure  and 
simple— age-old  resistance  to  change,  fear  of  the  unknown,  insecurity  {job,  ego,  etc.), 
etc. 
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4  Summary  of  Recommendations 


A  major  goal  of  this  workshop  was  to  foster  further  work  in  the  field,  directing  it  to  ends  that 
appear  fruitful.  To  this  end,  each  work  group  recommended  one  or  more  appropriate  actions.  In 
this  summary,  the  recommendations  are  considered  in  five  classes:  Improve  Contract  Models, 
Revise  Funding  Approaches,  Adapt  Acquisition  Policies,  Enhance  IPTs,  Provide  Training,  and 
Study  SD  and  EA.  The  full  recommendations  are  in  the  work  group  reports  in  Section  5  and 
are  collected  in  the  appendix.  Summaries  of  each  class  follow. 

Improve  Contract  Models 

Incorporation  of  EA  and  SD  in  projects  requires  new  contract  clauses,  new  kinds  of  contracts, 
and  new  procedures  for  letting  contracts.  The  Barriers  work  group  recommended  that  De¬ 
fense  Contract  Management  Agency  (DCMA)  and  OSD  develop  contract  models,  including 
even  suggested  wording  for  contract  performance  incentives.  There  should  also  be  incentives 
to  encourage  sharing  information  and  work  products  among  contractors.  In  a  cautionary  dis¬ 
sent,  the  Mapping  group  noted  that  SD  is  incompatible  with  programs  that  are  fixed  price  or 
use  strict  earned  value  tracking. 

Revise  Funding  Approaches 

The  Barriers  group  began  with  the  assertion  that  there  are  no  changes  needed  to  legislation  or 
regulation  in  order  to  allow  contract  innovation  that  will  encourage  adoption  of  SD  and  EA. 
Nonetheless,  the  group  suggested  several  innovations  that  should  occur  within  the  existing 
strictures.  For  instance,  budgets  should  have  funding  to  support  stakeholder  participation. 
The  central  need  is  more  funding  flexibility  because  the  risk  evaluation  in  each  spiral  cycle 
may  reveal  urgent  tasks  and  because  spiral  cycles  may  not  mesh  well  with  funding  cycles. 
The  needed  flexibility  may  be  acquired  for  C2  projects  by  combining  funding  at  the  PEO 
level  instead  of  fragmenting  it  down  to  the  PM  level.  Any  such  leeway  in  funding  approach 
will  be  unrecognizable  to  the  financial  management  community,  and  so  they,  too,  must  be 
educated  and  enrolled  in  the  drive  to  reach  the  advantages  of  EA  and  SD. 

Adapt  Acquisition  Policies 

Although  DoDI  5000.2  has  begun  a  process  of  adaptation  of  acquisition  policy,  the  work 
groups  suggest  that  the  task  is  not  yet  complete.  As  adaptations  proceed,  they  recommend 
that  changes  be  coordinated  by  a  single  office  or  IPT  and  that  this  group  prepare  a  plan  for 
rollout  of  and  transition  to  the  new  approaches.  In  all  this  plaiming  and  adaptation,  DoD 
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should  seek  the  advice  and  involvement  of  industry,  International  Council  on  Systems  Engi¬ 
neering  (INCOSE),  and  Project  Management  Institute  (PMI). 

The  Mapping  group  had  a  number  of  suggestions  in  terms  of  adaptation  of  the  EA  and  SD 
policies.  They  recommended  incorporating  robustness  and  adopting  anchor  point  milestones. 
They  suggested  that  EA  is  too  often  represented  as  a  linear  process  and  should  instead  be 
shown  in  its  true,  iterative  nature.  Finally,  they  pointed  out  the  necessity  of  distinguishing 
between  information  systems  (e.g.,  logistics  or  C2)  and  software  intensive  systems  (such  as 
weapons  acquisitions). 

Enhance IPTs 

A  number  of  suggestions  addressed  integrated  product  teams  (IPTs).  Their  number  should  be 
limited  to  enhance  their  power  and  reduce  the  number  of  personnel  commitments.  The  size  of 
each  must  be  kept  manageable,  and  yet  the  definition  of  stakeholder  needs  to  be  broadened  to 
include  representation  from  groups  such  as  oversight,  budget,  contractor,  contracting,  and  test 
organizations.  IPTs  must  be  given  budgetary  and  decision  making  authority.  They  must  have 
authority  to  adapt  requirements  and  set  testing  plans.  To  lend  this  authority  and  to  set  an  ex¬ 
ample,  personnel  at  the  PEO  level  should  be  members  of — and  active  participants  in — at  least 
the  highest  level  IPT.  Above  this  there  may  be  a  command  wide  “overarching”  IPT.  This 
group  should  focus  on  the  risk  management  plans  established  in  each  lower  level. 

Provide  Training 

People  need  new  skills  to  deal  with  EA  and  SD.  These  skills  can  come  from  post-secondary 
education,  service-wide  training,  and  training  within  specific  projects.  At  the  most  personal 
level,  the  work  groups  recommended  creation  of  a  cadre  of  gurus  to  aid  programs  in  adoption 
of  EAand  SD.  In  addition,  various  forms  of  written  instructional  material  must  be  developed, 
including  guidebooks  and  an  insert  on  the  definitions  of  EA  and  SD  to  accompany  copies  of 
DoDI  50(X).2.  At  the  educational  level,  modules  on  EA  and  SD  need  to  be  developed  and  in¬ 
corporated  into  the  curricula  of  Professional  Military  Education  (PME)  and  the  Defense  Ac¬ 
quisition  University  (DAU). 

Study  SD  and  EA 

SD  and  EA  require  further  study  in  order  to  validate  and  improve  the  processes.  Three  sepa¬ 
rate  work  groups  recommended  producing  documented  case  studies  of  EA  and  SD  projects. 
There  should  also  be  studies  of  completed,  non-EA  projects  to  determine  the  degree  to  which 
EA  would  have  improved  the  process.  In  addition  to  documenting  projects,  the  work  groups 
recommended  conducting  pilot  studies  to  explore  and  validate  the  methods.  These  studies  can 
incorporate  innovations  such  as  technology  to  continuously  involve  the  user  community.  One 
arena  for  pilot  studies  should  be  the  Simulation  Based  Acquisition  Initiative  (SBAI).  Finally, 
the  SD  process  needs  to  be  enhanced  with  a  catalog  of  risk  patterns  and  suggested  mitigation 
strategies. 
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In  his  opening  charge  to  the  conference.  Dr.  Steve  Cross,  Director  of  the  SEI,  asked  three 
questions.  (He  suggested  that  the  answers  might  be  “no.”)  In  summing  up  the  work  group 
presentations.  Cross  returned  to  these  questions  and  found  reasons  for  hope.  He  said 

•  Can  we  define  EA? 

Yes,  but  we  all  need  to  educate  our  communities— advocacy  needed 

•  Do  we  know  how  to  do  it? 

Some  of  us  do,  but  we  need  to  make  it  understandable  and  repeatable— need 
pilots,  case  studies,  training,  advisory  services,  etc. 

•  Will  we  do  it? 

It  is  up  to  us  to  make  change  happen. 

With  this  charge,  attendees  went  forth  to  engage  the  world. 
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5  Work  Group  Reports 


The  five  work  groups  were  each  assigned  a  specific  topic; 

Group  “Definition”:  Canonical  definition  of  EA 

Group  “Strategies”:  Successful  strategies  for  EA 

Group  “Mapping”:  Mapping  between  SD  and  EA 

Group  “Barriers”:  Institutional  barriers  to  EA  and  SD 

Group  “Operationalization”:  Practical  steps  to  operationalize  SD  and  EA 


Work  groups  were  asked  to  focus  on  producing  “actionable  recommendations”: 

•  Who  should  do  X? 

•  Who  needs  X?  (That  is,  who  should  be/could  be  the  customer  and  funder?) 

•  Why  is  it  needed  (rationale),  what  is  the  benefit? 

•  What  is  a  reasonable  time  frame  for  accomplishment  (and  how  much  might  it  cost)? 

Each  work  group  was  asked  to  cover  the  following  aspects  of  its  topic. 

•  A  Vision  Of  Successful  Spiral  Development  Practice 

•  Problem  And  Challenges  Facing  SDM 

•  Critical  Success  Factors  To  Success  Of  SDM 

•  Recommended  Actions 
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5.1  Work  Group  “Definition”:  Canonical  Definition  of 
Evolutionary  Acquisition 

Panel:  Pat  Place,  SEI  (Co-Chair);  Don  Reifer,  CSE  ( Co-Chair);  Clyde  Ellis, 

Quantum  Research;  Iris  Kameny,  Rand;  Jim  Linnehan,  Army;  Jim 
Marple,  SPC;  Gary  Thomas,  Raytheon 

Work  Group  Purpose 

The  purpose  of  this  working  group  was  to  define  evolutionary  acquisition  in  order  to 

•  Foster  a  common  understanding  of  evolutionary  acquisition 

•  Bring  clarity  to  policy  documents 

•  Serve  as  a  basis  for  promotion  of  the  evolutionary  acquisition  method 

The  working  group  wanted  to  produce 

•  A  crisp  definition  of  evolutionary  acquisition 

•  Standard  terminology  for  evolutionary  acquisition 

•  Characteristics  of  common  DoD  evolutionary  acquisition  variants  and  invariants 

Deliberations 

The  group  chose  to  tackle  the  issues  of 

•  Confusion  within  5000  documents  over  terms  evolutionary  /ncremenfa/ acquisition 
approaches 

•  Lack  of  a  characterization  of  the  approaches  that  PEOs  and  PMs  can  relate  to  when 
reading  the  documents 

Standard  brainstorming  was  chosen  as  a  working  process,  with  the  “right  hand”  rule  of  en¬ 
gagement:  when  two  wished  to  speak,  the  turn  passed  to  the  right  from  the  present  speaker. 

Our  major  conclusion  was  that  we  needed  to  expand  the  task  to  come  up  with  standard  defini¬ 
tions  for  both  Evolutionary  and  Incremental  Acquisition.  We  did  so  and  came  up  with  these 
definitions: 

•  Evolutionary  Acquisition  deploys  core  capability  and  incrementally  inserts  additional 
capabilities  as  requirements  are  refined. 

•  Incremental  Acquisition  deploys  a  full  capability  that  is  incrementally  fielded  based 
upon  firm  requirements  for  each  block. 

These  definitions  can  be  refined  with  invariants  and  variants.  Here,  an  invariant  is  an  ap¬ 
proach  that  is  invariably  followed  as  part  of  the  method,  and  a  variant  is  a  dimension  along 
which  the  process  can  be  varied  while  still  following  the  model.  The  invariants  and  variants 
for  the  two  acquisition  models  are  listed  and  compared  in  Table  2. 
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Table  2:  Invariants  and  Variants  for  EA  and  lA 


Evolutionary  Acquisition 

Incremental  Acquisition 

1.  Multiple  deployments  accommodating 
evolving  requirements 

Opportunistic  technology  insertion 

Many  unknoivns 

Evolving  threats 

Users’  understanding  of  delivered 
capabilities 

Related  variants:  Degree  of  flexibility; 

Degree  of  time  phasing 

1.  Requirements  for  multiple 

deployments  are  firmly  defined 

Planned  technology  insertion 

Few  unknowns 

Defined  threat 

Users’  understanding  of  final  capabilities 
Related  variant:  Degree  of  time  phasing 

2.  Risk  driven 

Increments  defined  to  reduce  risks 

Risks  influence  increment  content 

Related  variant:  Unknown  number  of 
increments 

2.  Requirements  driven 

Capabilities  for  each  increment  are  defined 

Related  variant:  Known  number  of 
increments 

3.  Emphaas  on  toU 

Can  plan  as  you  go 

All  life  cycle  activities  must  b 
Related  variants:  Testing,  trair 

il  life  cycle  activities 

Preplanned 

e  planned  for  each  increment 

dng,  support,  etc. 

4.  Decision  point  at  end  of  each  increment  involving  appropriate  stakeholders 

“Field”  or  “No  Field”  decision;  “Am  I  Done”  decision 

Related  variants:  List  of  appropriate  stakeholders;  Choice  of 

decision  making  method  (risk-based,  funding,  etc.) 

The  resulting  definitions  differ  from  those  offered  by  the  February  workshop  in  several  ways. 
First,  the  invariants  have  been  expanded  by  listing  the  characteristics  of  each.  Second,  two 
February  invariants — concerning  deployment  and  requirements — have  been  combined  into 
Invariant  1  above.  Third,  Incremental  Acquisition  has  been  introduced  and  defined.  Fourth, 
references  to  Anchor  Point  Milestones  were  removed  because  they  apply  during  development 
rather  than  during  acquisition.  (Their  place  should  be  addressed  by  the  Mapping  work 
group.) 

Recommendations 

The  group  made  the  following  recommendations.  In  order  to  be  concrete,  each  specifies  the 
office  of  primary  responsibility  (“agent”)  and  a  date  by  which  the  work  can  and  should  be 
accomplished. 
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•  Use  definitions  to  develop  an  insert  that  can  be  put  into  5000  Directives,  Regulations, 
and  Instructions.  Agent;  SIS  Office.  Due  date;  31  October  2000. 

•  Develop  implementation  guidance  for  existing  publications,  e.g.,  API  63-123. 

Agent;  SIS  Office.  Due  date;  30  April  2001. 

•  Develop  guidebook  for  PMs  and  PEOs  addressing  “top  25”  implementation  ques¬ 
tions.  Agent;  SIS  Office.  Due  date;  30  April  2001. 

•  Document  experiential  case  studies.  Agent;  SIS  Office.  Due  date;  30  April  2001. 

•  Identify  focal  point  in  SIS  Office  for  EA.  Agent;  SIS  Office.  Due  date;  30  April 

2001. 

•  Develop  education  and  training  modules  for  insertion  into  Acquisition  Management 
courses.  The  appropriate  level  of  detail  is  a  two-hour  module  explaining  use  of  EA 
and  a  four-hour  module  on  case  studies.  Agent;  DSMC.  Due  date;  1  September 
2001. 
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5.2  Work  Group  “Strategies”:  Successful  Strategies 
for  Evolutionary  Acquisition 

Panel:  Lisa  Brownsword,  SEI  ( Co-chair);  Caroline  Graettinger,  SEI  (Co¬ 

chair);  Robert  Flowe,  OSD/PA&E;  Charles  Leinbach,  C-bridge; 
Bobby  McKinnon,  Army  War  College;  Jeff  Rothenberg,  Rand 

Work  Group  Purpose 

This  purpose  of  this  working  group  was  to  capture  the  participants’  experiences  in  the  use  of 
evolutionary  acquisition  (EA)  in  order  to 

•  provide  guidelines 

•  collect  successful  strategies  to  aid  others  in  successful  use  of  EA 

While  the  focus  of  the  working  group  was  EA,  the  participants  were  using  spiral  develop¬ 
ment  (SD)  as  well.  Due  to  work  group  time  limitations,  descriptions  and  conclusions  are 
necessarily  preliminary. 

Participants 

The  work  group  participants  included 

•  Lisa  Brownsword,  SEI,  co-lead 

•  Robert  Flowe,  OSD/PA&E  (cost  analysis);  previous  experience  with  ESC  and  Air 
Intelligence  Agency  to  implement  a  more  efficient  acquisition  approach  using  spiral  de¬ 
velopment;  developed  acquisition  strategy  for  an  ACAT  IE  program 

•  Caroline  Graettinger,  SEI,  co-lead 

•  Charles  Leinbach,  C-bridge;  experience  leading  customers  through  EA  process  to  sup¬ 
port  C-bridge  SD  process;  worked  with  40  e-business  systems,  $0.5-10M  sized  pro¬ 
jects;  authored  large  part  of  C-bridge’s  RAPID  Value™  method 

•  Bobby  McKinnon,  Army  War  College  (senior  service  fellow);  previous  experience  as 
product  manager  for  ACAT  1AM  program;  evolutionary  and  incremental  acquisition 
program,  C4I  systems 

•  Jeff  Rothenberg,  Rand,  doing  study  on  EA 

Working  Definitions 

While  other  groups  were  tasked  to  form  operative  definitions  for  EA  and  SD,  this  group 
found  the  need  for  a  common  frame  of  reference  for  strategy  discussions: 

The  focus  of  EA  is  to  field  a  core  capability  quickly  and  then  add  capability  based  on 
user  input  and  priorities  over  time.  With  EA,  users  don’t  know  exactly  what  is  needed  in 
the  system.  EA  requires  some  general  notion  of  system  scope  and  boundaries  at  the  op¬ 
erational  level  (i.e.,  an  operational  architecture)  before  it  is  begun. 

SD  is  one  development  approach  for  satisfying  the  requirements  for  EA. 

SD  and  EA  are  separate  processes  but  share  some  underlying  characteristics,  such  as 
being  risk-driven,  architecture-focused,  and  engaging  continuous  user  involvement. 
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Successful  Strategies 

The  working  group  explored  three  EA/SD  approaches.  The  experience  of  the  group  did  not 
include  embedded  software  systems. 

5.2.1  Approach  A:  COTS  +  GOTS  +  Custom  Software  Devel¬ 
opment 

Motivation  for  EA/SD 

There  was  a  lack  of  clarity  in  the  overall  requirements  base,  which  motivated  the  use  of  evo¬ 
lutionary  acquisition.  Program  participants  knew  they  had  funding  limitations  that  argued  for 
an  incremental  approach  in  those  areas  where  the  requirements  were  somewhat  better  de¬ 
fined. 

Summary  of  Approach 

This  approach  identified  a  fielded  basic  capability  based  on  the  input  of  the  users  and  limited 
by  available  resources.  The  program  created  the  infrastructure  (for  example,  office  automa¬ 
tion,  network,  desktops)  in  the  first  increment.  Use  defined  each  new  increment  that  was 
added  in  roughly  six-month  increments.  Successive  increments  developed  the  applications 
that  worked  on  the  infrastructure  fielded  in  the  basic  capability.  Respected  stakeholders  par¬ 
ticipated  so  all  their  requirements  were  addressed.  Maintaining  user  involvement  through 
incentives  was  also  a  key  factor. 

Multiple  contract  vehicles,  such  as  Indefinite  Delivery  Indefinitive  Quantity  (IDIQ),  level  of 
effort,  and  time  and  materials,  were  used  to  balance  the  risk  between  the  government  and  the 
contractor.  The  core  capability  used  an  IDIQ  contract.  Integration  and  testing  were  done 
with  a  time  and  materials  contract.  Proposals  identified  labor  categories  and  total  hours  for 
government  comparison  and  negotiation  with  the  contractor.  The  program  used  a  fixed  price 
contract  to  manage  and  develop  the  application  capability  where  requirements  were  well 
known  and  implementation  was  low  risk.  The  contract  would  limit  total  hours  and  was  set  up 
each  year  based  on  contractor  input.  Award  fees  were  a  useful  device  for  increasing  customer 
satisfaction. 

Key  Strategies 

The  following  strategies  were  considered  key  in  applying  Approach  A. 

•  Identify  and  field  a  basic  capability  that  was  defined  based  on  user  input  within  the 
context  of  available  resources. 

•  Define  the  next  increment  of  capability  based  on  negotiation  between  users,  develop¬ 
ers,  testers,  etc. 

•  Add  additional  capability  in  increments  of  roughly  six  months. 

•  Use  multiple  contract  vehicles  to  balance  the  risk  between  the  government  and  the  con¬ 
tractor  (IDIQ,  level  of  effort,  time  and  materials). 
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•  Use  award  fees  to  provide  incentives  for  the  contractor. 

•  Provide  incentives  for  the  user. 

5.2.2  Approach  B:  C4I  Program,  ACAT  III 
Motivation  for  EA/SD 

The  members  of  this  C4I  program  considered  that  the  biggest  risk  was  the  anticipated  major 
change  in  concept  of  operations.  Therefore  they  chose  an  EA  and  SD  approach  that  strove  to 
include  constant  user  involvement. 

Summary  of  Approach 

Work  with  stakeholders  of  all  affected  areas  (including  security  and  interoperability)  to  derive 
a  wish-list  of  features.  Prioritize  the  list  in  collaboration  with  the  stakeholders  and  perform  a 
risk  analysis  of  each  item  to  form  a  priority/risk  matrix.  Address  high-priority  requirements 
up  front.  While  all  high-priority  items  may  not  be  incorporated  into  the  first  build,  a  program 
must  begin  allocating  resources  to  address  them  from  the  beginning.  High-risk  items  are  ex¬ 
plored  through  short  duration  prototypes  that  users  then  evaluate  to  better  define  the  require¬ 
ments.  Prototypes  are  also  used  to  explore  technical  feasibility  of  an  implementation.  Archi¬ 
tecture  constraints  are  identified  concurrently  with  requirements  elicitation/analysis. 

Stakeholders  and  the  program  office  allocate  prioritized  wish/requirements  list  items  to  up¬ 
coming  increments.  An  overarching  procurement  and  management  plan  is  created  that  deter¬ 
mines  increments,  including  cost,  schedule,  and  general  functionality.  Increment  delivery 
schedules  are  based  on  how  rapidly  the  user  community  incorporates  the  release  into  its  op¬ 
erations. 

A  vital  concern  for  program  management  is  obtaining  end-user  involvement  when  users  are 
trying  to  deliver  on  their  own  missions  and  retaining  the  involvement  of  experienced,  knowl¬ 
edgeable  users.  This  program  was  able  to  co-locate  the  acquisition  group  with  end  users  or 
end-user  surrogates.  This  includes  funding  for  the  involvement  of  the  end  users. 

Day-to-day  interactions  are  necessary  so  acquisition  folks  can  absorb  the  requirements  and 
priorities  of  the  users.  This  also  helps  facilitate  dialogue  to  help  users  understand  the  impli¬ 
cations  of  what  they  are  asking  for  and  what  their  expectations  ought  to  be.  Setting  the  expec¬ 
tations  of  end  users  for  any  prototypes  must  be  carefully  managed.  Feedback  from  users  is 
well  structured  to  minimize  impact  to  their  normal  mission  work. 

Flexible  contracting  strategy  is  required  to  allow  risk  mitigation.  A  combination  of  contract 
vehicle  types,  such  as  time  and  materials  or  task  orders,  are  vital,  including  the  use  of  appro¬ 
priate  contractor  incentives. 
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Key  Strategies 

•  Users  or  user  surrogates  are  critical.  Make  it  important  to  them  to  participate.  A  pro¬ 
gram  will  need  robust,  high-bandwidth  access  to  users. 

•  Be  judicious  about  your  release  cycles;  don’t  perturb  the  users  with  too  frequent  re¬ 
leases.  Involve  the  users  in  planning  for  release  cycles  including  training,  rollout,  data 
conversion,  and  organizational  change  impacts. 

•  Continuous  engagement  with  the  stakeholder  community  may  drive  system  engineer¬ 
ing  and  program  management  resource  and  skill  requirements,  so  don’t  scrimp  on  staff. 
The  program  management  staff  must  have  greater  skills  than  what  is  typical  now. 

•  Use  flexible  contracting  (time  and  materials,  task  orders,  appropriate  incentives)  to  al- 
low  risk  mitigation. 

•  Address  the  high-priority  items  first,  don’t  defer  the  high-risk  items. 


5.2.3  Approach  C:  e-Business  Systems 

Motivation  for  EA/SD 

This  is  the  standard  acquisition  and  development  approach  this  consulting  firm  uses  to  men¬ 
tor  other  programs  in  developing  e-business  systems. 

Summary  of  Approach 

The  acquisition  process  starts  with  a  planning  phase  that  explicitly  defines  objectives. 

A  Visioning  Session,  involving  the  major  stakeholders — and  especially  end  users — is  held  to 
help  develop  a  preliminary  list  of  featmes.  In  parallel,  the  business  value  (economics)  of  the 
project  is  translated  into  budget  estimates  and  a  plan  for  spirals  in  the  project  development.  A 
priority/risk  matrix  is  created  describing  what  the  program  should  be  doing  to  meet  its  strat¬ 
egy.  As  the  project  proceeds,  risks  are  revisited.  The  preliminary  architecture  is  designed  by 
the  end  of  the  planning  phase.  A  key  element  of  the  architecture  is  its  growth  capacity  across 
increments. 

The  next  phase  is  the  first  EA  increment  that  results  in  an  increment  delivery.  Every  incre¬ 
ment’s  goal  should  be  delivery  into  real-world  environments.  The  infrastructure  needed  by 
the  end  user  is  delivered  in  the  earliest  increments.  The  first  release  incorporates  tasks  from 
the  ends  of  the  list:  the  easiest  because  they  can  be  done  and  the  hardest  in  order  to  mitigate 
the  risks  they  pose.  Management  of  configurations  and  changes  is  required  from  the  outset  of 
the  project.  The  configuration  and  change  control  process  must  be  well  defined  and  effi¬ 
ciently  practiced.  Weekly  meetings  to  look  at  the  artifacts  of  the  projects  and  new  risks  are 
also  required.  In  early  increments,  development  changes  are  relatively  easy  to  make  and  re¬ 
quire  low-level  approval  authority.  In  later  increments,  higher  levels  of  approval  are  re¬ 
quired. 
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Key  Lessons 

•  As  part  of  the  project  preparation,  educate  the  customer,  the  user,  and  the  contractor  on 
the  EA  approach. 

•  Goal  of  every  EA  increment  should  be  delivery  into  a  real-world  environment. 

•  In  the  first  increment,  put  into  place  the  infrastructure  that  the  end  user  and  the  devel¬ 
opment  team  will  need. 

•  Create  a  very  well  defined  configuration  and  change  control  process  at  the  outset. 

•  Design  the  primary  architecture  by  the  end  of  planning  phase. 

•  Make  sure  there  is  a  strong  chief  technical  architect  in  the  program  office. 

•  At  the  project  outset,  define  the  goals  of  the  project  with  active  user  involvement. 

5.2.4  Strategy  Summary  from  All  Approaches 

From  the  three  EA/SD  approaches  that  the  working  group  explored,  the  following  essential 
tactics  were  extracted: 

•  Develop  an  acquisition  strategy  that  allows  you  to  address  high-risk,  high-value  areas 
first.  Don’t  defer  high-risk  elements  to  the  end.  Start  allocating  resources  towards 
high-risk  elements. 

•  Build  something  that  is  fielded  and  usable  at  every  increment  that  includes  adequate 
documentation,  training,  and  support. 

•  Balance  risks  between  stakeholders  through  a  team  environment  that  shares  risks  and 
tools. 

•  Develop  incentives  for  stakeholders  to  remain  engaged. 

•  Make  sure  you  understand  the  motivations  of  each  stakeholder  group. 

•  Avoid  mandating  a  single  life-cycle  model  -  one  size  does  not  fit  all. 

•  Contract  vehicles  are  important  tools  in  flexibility.  Allow  the  use  of  all  contract  vehi¬ 
cle  types,  where  applicable  (e.g.,  fixed  price,  time/materials). 

•  Provide  appropriate  incentives  for  all  stakeholders. 

•  Hold  people  and  organizations  accountable. 

Maintaining  management  involvement  is  key.  EA  is  not  a  license  to  abdicate  responsibilities. 
Maintain  management  involvement  through  frequent  reviews  that  involve  all  stakeholders. 

Recommendations  for  Further  Work 

The  working  group’s  personal  experience  does  not  comprise  a  statistically  significant  sample. 
As  a  result,  we  recommend 

•  Development  and  dissemination  of  detailed  case  studies  of  EA  and  SD.  Potentially  this 
work  could  be  done  through  the  sponsorship  of  C3I  and  AT&L. 

Group  members  found  from  their  collective  experience  that  creating  and  sustaining  the  par¬ 
ticipation  of  relevant  stakeholder  communities,  in  particular  end  users,  was  foundational  for 
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successful  EA  and  SD  approaches,  but  there  are  limited  codified  practices.  Thus,  we  also 
strongly  recommend  that  the  DoD 

•  Identify  and  prototype  effective  practices  and  supporting  technologies  that  facilitate  the 
sustained  engagement  of  the  stakeholder  communities,  and  especially  end  users. 

Lastly,  we  recommend 

•  Development  of  a  cadre  of  industry  and  government  people  knowledgeable  in  the 
pragmatic  application  of  EA  and  SD  who  are  readily  available  to  program  offices  and 
contractors. 
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5.3  Work  Group  “Mapping”:  Mapping  Between  Spiral 
Development  and  Evolutionary  Acquisition 

Panel:  Elliot  Axelband,  USC  ( Co-Chair);  David  Carney,  SEI  ( Co-Chair); 

Brad  Clark,  SEI;  Kevin  Forsberg,  CSM;  Elwyn  Harris,  Rand;  Katy 
Smith,  MITRE 

This  group  was  chartered  to  consider  the  relationship  between  spiral  development  (SD)  and 
evolutionary  acquisition  (EA).  This  section  generally  follows  the  attached  viewgraphs  that 
were  used  to  sunuxiarize  the  group’s  conclusions.  These  were  presented  on  the  final  day  of 
the  workshop. 

A  framework  was  developed  during  the  workshop  that  provides  the  context  of  this  section 
and  which  is  essential  to  understanding  it. 

Framework 

The  context  of  the  workshop  was  software  intensive  systems.  The  natural  question  then  is 
what  is  a  software  intensive  system?  While  the  “Definition”  Work  Group  formally  addressed 
this  question  during  its  working  session,  the  framework  was  established  by  the  workshop  as  a 
whole,  prior  to  the  convening  of  the  work  groups  and  their  subsequent  reports. 

The  discussion  was  driven  by  asking  whether  the  newly  planned  Navy  surface  combatant 
DD-21,  the  multi-service  next  fighter  aircraft  -  JSF  (Joint  Strike  Fighter),  and  the  Army’s 
next  combat  vehicle  -  FSCV  (Future  Scout  Combat  Vehicle),  were  examples  of  software  in¬ 
tensive  systems.  By  a  show  of  hands,  the  vote  was  almost  unanimous  that  they  were.  This  led 
to  the  conclusion  that  software  intensive  systems  were  really,  in  most  cases,  hardware,  peo¬ 
ple,  and  software  intensive,  a  viewpoint  that  must  be  retained  in  considering  the  conclusions 
of  this  and  other  working  groups. 

There  are  exceptions  to  this.  These  are  systems  that  are  only  housed  in  hardware  (computers) 
and  are  otherwise  pure  software.  These  go  under  various  guises:  C4ISR  systems,  information 
systems,  and  software-only  systems.  Such  systems,  however,  though  an  important  class,  have 
very  different  properties  than  the  more  general  class  of  systems  combining  hardware,  soft¬ 
ware,  and  people  that  are  considered  under  the  term  software  intensive  systems.  One  other 
term  of  art  used  by  one  member  of  the  C4ISR  community  was  to  call  non-C4ISR  systems  by 
the  term  “iron  systems”  which  was  his  way  of  attaching  a  distinctive  label  to  the  more  gen¬ 
eral  class  of  software  intensive  systems. 

The  question  of  whether  the  term  software  intensive  systems  included  hardware,  people,  and 
software  intensive  systems  such  as  JSF,  DD-21,  and  FSCV  was  also  addressed  independently 
on  the  last  day  of  the  conference  by  Dr.  Jack  Ferguson  of  the  DoD.  He  stated  that  they  were 
included,  confirming  the  interpretation  taken  by  the  workshop  as  a  whole. 
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A  distinction  of  note  was  the  composition  of  the  workshop — only  about  5%  of  those  present 
were  from  industry.  The  conference  was  government-  (including  FFRDCs)  and  university- 
dominated.  It  was  generally  agreed  that  such  a  skewed  composition  would  not  constitute  a 
good  IPX  and  would  be  remedied  at  future  related  events. 

A  disagreement  of  note  between  the  conclusions  of  this  working  group  and  the  “Barriers” 
working  group  concerns  the  existence  of  legal  barriers  to  the  incorporation  of  spiral  processes 
within  evolutionary  acquisition.  The  “Barriers”  working  group  reported  at  the  workshop  that 
here  are  none.  This  working  group,  however,  believes  that  under  existing  DoD  policy,  soft¬ 
ware  intensive  systems  that  contain  hardware  must  undergo  a  design  freeze  prior  to  the  start 
of  manufacture  and  so  must  suspend  spiral  processes  for  the  production  lot  under  manufac¬ 
ture.  Evolutionary  acquisition  does  not  preclude  this;  nor  should  it,  given  the  capital  intensive 
nature  of  manufacturing.  An  independent  inquiry  subsequent  to  this  workshop  confirmed  the 
existence  of  that  DoD  policy. 

Summary 

Evolutionary  acquisition  as  embodied  in  the  current  versions  of  draft  DoD  5000.1  and  5000.2 
are  strongly  supported.  They  are  an  incremental  improvement  to  their  predecessor  documents 
and  incorporate  changes  that  hold  the  promise  of  improving  the  acquisition  process. 

Spiral  development  processes  have  the  potential  to  enhance  evolutionary  acquisition  as  an 
implementing  procedure  provided  they  are  incorporated  properly  and  with  due  consideration 
of  both  their  strengths  and  weaknesses  for  the  applications  being  considered.  Software  only 
systems  and  (hardware  and  people)  software  intensive  systems  are  very  different  in  their  na¬ 
ture  and  must  be  treated  as  two  separate  cases.  Procedures  that  proved  successful  in  one  case 
should  not  automatically  be  presumed  to  be  successful  in  the  other. 

Policies  regarding  the  incorporation  of  spiral  development  should  evolve  from  interaction 
among  the  players  in  a  fully  representative  IPT.  Government,  universities,  and  industry 
should  all  be  represented. 

Policies  should  not  be  imposed  by  fiat  or  on  the  basis  of  anecdotal  evidence.  The  use  of  stud¬ 
ies,  test  cases,  and  prototypes  is  advocated  to  establish  a  minimally  intrusive  body  of  evi¬ 
dence  to  serve  as  basis  for  policy.  More  generally,  policy  makers  are  urged  to  lead  by  exam¬ 
ple,  and  not  fiat,  where  possible. 

Within  government  there  are  numerous  established  policies  and  instituted  behaviors  that  are 
antithetical  to  spiral  processes  and  which  must  be  addressed.  Many  of  these  have  been  en¬ 
countered  as  factors  that  have  limited  the  success  realized  by  acquisition  streamlining  these 
past  five  years.  They  include  (1)  rapid  rotation  of  assignments  of  military  acquisition  officers, 
(2)  program  cancellation  as  a  mark  of  failure,  and  (3)  the  government  contracting  commu¬ 
nity’s  aversion  to  risk-type  contracts.  These  are  discussed  more  extensively  in  the  body  of 
the  section. 
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Further,  there  is  not  a  common  understanding  of  the  flexibility  inherent  in  waterfall,  V,  Syn¬ 
chro  X,  and  other  models  of  development  processes  and  in  the  properties  of  production  proc¬ 
esses.  This  limits  the  potential  for  success  that  will  be  achieved  in  incorporating  spiral  proc¬ 
esses.  Further  interaction  and  improved  BPTs  will  provide  the  common  understanding  that 
will  accelerate  the  well-thought-out  introduction  of  spiral  development  into  evolutionaiy  ac¬ 
quisition. 

Mission 

This  working  group  was  chartered  to  understand  the  interaction  between  spiral  development 
and  evolutionary  acquisition  in  order  to  clarify  the  various  government  and  contractor  roles. 

Vision  of  2010 

In  an  ideal  world  in  the  year  2010  the  acquisition  community  would  be  well  trained  in  evolu¬ 
tionary  acquisition  and  the  array  of  strategies — including  spiral  development  to  implement 
it.  Strategies  would  be  employed  as  a  function  of  the  nature  of  the  program  under  considera¬ 
tion  drawing  upon  the  broad  experience  gained  in  past  applications.  Strategists  would  have  a 
good  understanding  of  the  complementary  as  well  as  contrasting  nature  of  various  strategies 
in  order  to  make  well-considered  decisions.  Such  understanding  will  include  the  facts  that 
spiral  development  is  not  incompatible  with  waterfall  processes,  and  that  spiral  development 
is  incompatible  with  the  fixed  price  contracting  approach  that  is  required  for  any  one  produc¬ 
tion  lot. 

Congress,  the  DoD,  the  services,  and  industry  would  understand  that  acquisition  strategy 
must  be  tailored  to  the  nature  of  the  program  to  which  it  is  applied;  they  would  be  generally 
supportive  of  this  approach  and  of  evolutionary  acquisition.  All  participating  parties  would  be 
incentivized  to  take  a  tailored  evolutionary  acquisition  approach  and  would  be  rewarded  upon 
success  in  the  form  of  professional  advancement  and— for  industry— incentive  fees.  Metrics 
would  be  clearly  established  to  provide  the  basis  of  evaluating  performance,  including  sched¬ 
ule,  cost,  performance,  technology  insertion,  international  cooperation,  standardization,  and 
risk  reduction,  as  appropriate.  It  would  be  recognized  that  program  cancellation  is  a  reason¬ 
able  occurrence  when  undertaking  high-risk  programs,  and  rewards  would  be  provided  for 
the  correct  early  determination  that  cancellation  is  the  proper  action. 

The  DoD’s  processes  related  to  evolutionary  acquisition  would  be  supportive.  Program  man¬ 
agers  would  be  long  tenured  so  that  they  can  gain  experience  throughout  the  considerable 
program  length  some  programs  would  require.  DoD  program  managers  would  be  rewarded 
for  program  cancellation  when  that  is  a  proper  decision  in  the  interest  of  the  DoD.  Contract¬ 
ing  officers  would  be  prepared  to  offer  and  endorse  flexible  contracting  methods  when  these 
are  required  to  implement  spiral  development. 

Congress  would  recognize  the  need  to  stabilize  the  defense  budget  and  this  in  turn  will  pro¬ 
vide  a  stable  level  of  employment  for  defense  professionals  that  will  enhance  their  ability  to 
leam  and  apply  successful  evolutionary  acquisition  techniques. 
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Two  Types  of  Systems 

Key  to  the  success  of  evolutionary  acquisition  and  the  use  of  spiral  development  as  an  im¬ 
plementing  tool  is  the  ability  to  distinguish  between  two  very  different  types  of  systems  that 
are  covered  under  the  rubric  of  software  intensive  systems.  By  software  intensive  systems 
(called  by  some  “iron  systems”),  we  generally  mean  those  that  have  significant  hardware  and 
people  content.  By  information  systems  (called  by  some  C4ISR  systems)  we  mean  those 
where  the  hardware  is  generally  a  commercial  off-the  shelf/Govemment  off-the-shelf 
(COTS/GOTS)  computer,  housing  software  that  is  the  essence  of  the  system. 

Information  systems  have  no  production  phase  in  their  development,  can  be  relatively  easily 
changed  after  deployment  by  the  use  of  patches,  and  can  have  great  synergy  with  commercial 
software  developments. 

Software  intensive  systems,  by  contrast,  have  a  long  production  cycle,  are  relatively  difficult 
to  change  after  deployment,  and  have  less  synergy,  except  perhaps  at  the  electronic  compo¬ 
nent  level. 

These  differences  are  profound  and  must  be  regarded  with  care.  Nonetheless,  increased  use  of 
spiral  development  can  enhance  evolutionary  acquisition  in  both  cases. 

Anticipated  Difficulties 

In  the  year  2010,  evolutionary  acquisition  enhanced  by  spiral  development  will  have  a  greater 
degree  of  concurrent  research  and  development,  production,  and  operation  for  a  given  pro¬ 
gram  than  occurs  today.  There  will  also  be  a  more  rapid  change  cycle.  While  these  are  desir¬ 
able  outcomes  that  derive  from  evolutionary  acquisition  and  spiral  development,  certain  dif¬ 
ficulties  will  be  encountered  during  implementation  and  they  must  be  anticipated  and 
included  in  program  planning.  These  are  as  follows. 

•  increased  simultaneous  use  of  colors  of  money,  and  ambiguity  in  the  proper  application 
of  colors  of  money 

•  a  greater  coupling  with  the  annual  budget  cycle 

•  more  resources  to  understand  end-user  feedback 

•  a  greater  need  for  upward  and  downward  compatibility  of  deployed  systems 

•  a  greater  acceptance  of  risk  and  tolerance  for  failure  by  the  acquisition  community 

•  an  increased  need  for  frequent  all-player  program  reviews  and  the  funds  to  support 
them 

Recommendations 

We  recommend  the  following  steps  to  work  toward  the  improvement  and  adoption  of  evolu¬ 
tionary  acquisition; 

•  Convene  an  IPT  to  further  the  incorporation  of  SD  within  EA.  In  addition  to  DoD  per¬ 
sonnel,  membership  should  include  representatives  from  industry,  INCOSE,  and  PMI 
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who  will  involve  their  home  organizations  in  furthering  the  work  (Ferguson,  with  sup¬ 
port  from  Boehm,  Foreman,  and  Axelband). 

•  Explore  EA  further  prior  to  imposing  it  as  a  policy.  Evaluate  how  current  programs 
would  have  fared  under  EA.  Conduct  a  prototype  study  (Etter  with  SAEs). 

•  Plan  each  EA/SD  project  so  consecutive  spiral  cycles  converge  toward  a  final  goal. 
These  plans  must  be  cognizant  of — ^and  resilient  to — ^program  re-direction,  technology 
change  and  other  inevitable  sources  of  distraction.  Among  suitable  techniques  are  the 
use  of  anchor  points  advocated  by  Boehm  and  stable  intermediate  points  as  advocated 
by  Rechtin  [Boehm  00,  Rechtin  00  p.  99]. 

•  Depict  EA  so  as  to  reveal  the  feedback  possibilities  instead  of  making  it  appear  as  a 
rigid  linear  approach.  The  depiction  of  waterfall-like  processes  as  being  spiral-cycle 
free  is  inaccurate  and  misleading.  It  builds  barriers  to  understanding  the  full  tool  kit 
that  the  EA/SD  practitioner  can  use. 

•  Avoid  SD  for  the  manufacture  of  any  lot  in  programs  with  production  phases.  Produc¬ 
tion  programs  are  fixed  price  and  use  strict  earned  value  tracking,  both  of  which  are 
incompatible  with  SD. 

•  In  applying  EA/SD,  distinguish  between  information  systems  and  software  intensive 
systems. 


CMU/SEI-2001-SR-005 


37 


5.4  Work  Group  “Barriers”:  Institutional  Barriers  to 
Evolutionary  Acquisition  and  Spiral  Development 

Panel:  Ceci  Albert,  SEI  (Co-chair);  Fred  Hansen,  SEI  (Co-chair);  Tom  Bos- 

telaar,  TRW;  Dave  Durfee,  TASC;  Scott  Henninger,  U.  Neb;  KC  King, 
SAIC;  Stan  Levine,  DSCOPS-FD;  Warren  Morrison,  SEI 

Summary  of  Discussion 

This  work  group  determined  that  there  are  no  legal,  policy  or  regulatory  barriers  to  EA/SD. 
(After  all,  there  are  programs  that  have  successfully  applied  these  concepts.)  The  group 
firmly  believes  that  program  managers  already  have  the  latitude  to  implement  EA/SD — if  the 
program  manager  sees  EA/SD  as  a  solution  to  the  program’s  needs.  Each  program  manager 
must  select  the  most  appropriate  approach  to  address  unique  program  risks.  In  order  to  be 
successful  in  selecting  the  right  approach,  the  program  managers  must  understand  the  avail¬ 
able  alternatives  and  the  implications  of  each  alternative.  However,  the  culture  (the  overse¬ 
ers,  the  program  support  activities,  the  staffs)  must  change  to  accept  and  implement  the  pro¬ 
gram  manager’s  chosen  approach. 

On  the  other  hand,  the  group  was  concerned  that  the  amount  of  information  a  program  man¬ 
ager  would  need  to  know  to  successfully  select  and  implement  an  EA/SD  approach  might  be 
more  than  could  be  expected  of  a  “typical”  program  manager.  The  program  manager  must  be 
able  to 

•  write  time-phased  requirements  that  meet  cost  targets  by  increment 

•  define  the  architecture  up-front  (before  all  the  requirements  are  defined) 

•  maintain  credibility  and  support  for  a  program  with  flexible  requirements,  requiring 
flexible  contract  vehicles  and  fimding  strategies 

•  develop  the  analytic  basis  to  set  up  the  program 

•  know  the  program  risk  patterns  and  the  models  they  generate  and  which  need  EA/SD 

•  negotiate  appropriate  user  interfaces  in  an  environment  that  demands  active  stakeholder 
negotiation  of  flexible  requirements 

•  decide  what  is  good  enough  and  when  to  stop  development 

•  develop  metrics  and  incentives  that  support  EA/SD  implementation  in  support  of  pro¬ 
gram  goals. 

The  group  discussed  different  types  of  risk  patterns  that  must  be  accommodated  by  acquisi¬ 
tion  and  development  approaches.  For  example,  a  weapon  system  (such  as  a  tank)  requires  a 
huge  capital  investment  in  hardware  and  manufacturing  capability  to  produce,  and  the  re¬ 
quirements  are  relatively  easy  to  bound.  A  software  system  (such  as  a  command  and  control 
system),  however,  is  intrinsically  linked  to  the  demands  of  its  users;  therefore,  the  system  is 
linked  to  its  user  interface.  The  requirements  for  a  software  system  are  harder  to  determine  at 
any  point  of  time  than  a  weapon  system’s  requirements,  and  the  concept  of  operation  is  con¬ 
stantly  changing  across  the  life  of  the  system.  In  the  case  of  software  systems,  physical  pro- 
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totypes  are  easier  to  build  than  an  activity  model  of  the  system.  In  many  cases,  requirements 
can  be  characterized  as  “I’ll  know  it  when  I  see  it”  (DCIWISI). 

Because  of  the  experience  of  the  group  members,  the  group  concentrated  on  command  and 
control  type  software  systems. 

Vision 

For  2010,  the  work  group  envisions  an  environment  where 

•  each  project  follows  the  process  model  best  suited  to  its  unique  risks 

•  different  patterns  of  behavior  are  followed  based  on  an  understanding  of  the  pro¬ 
gram’s  unique  pattern  of  risks 

•  services  are  able  to  acquire  the  capabilities  they  need  in  an  era  of  diminishing  funds 
In  order  to  realize  this  vision,  program  managers,  their  staffs,  their  contracting  partners,  and 
the  end  users  must  be  trained,  skilled,  experienced,  and  accepting  of  risk  and  innovation; 
there  must  be  a  high  level  of  trust  and  cooperation  among  all  stakeholders  (end  users,  busi¬ 
ness  process  owners,  acquirers,  developers,  integrators,  vendors,  testers,  etc). 

Barriers 

Having  determined  that  there  are  no  formal  legal  or  policy  barriers,  the  group  identified  four 
major  categories  of  impediments  to  implementing  EA/SD.  The  group  characterized  some  of 
the  issues  associated  with  each  category  and  then  developed  specific  recommendations  aimed 
at  addressing  those  issues. 

The  first  category  is  risk  aversion.  This  category  includes  the  failure  to  use  risk  to  drive  the 
acquisition  and  development  processes  as  well  as  program  management;  the  failure  to  find 
and  identify  the  real  program  risks;  and  the  perceived  end-user  risk  of  receiving  a  first  incre¬ 
ment  that  has  less  capability  than  the  legacy  system  it  replaces.  Three  recommendations  were 
made  in  this  category: 

•  Senior  leadership  (PEO  and  comparable  end-user  representative)  should  demonstrate 
support  for  EA/SD  through  participation  in  spiral  development  integrated  product 
teams  (SDIPT). 

•  The  overarching  integrated  product  team  (DIPT)  should  focus  program  oversight  on 
the  program  manager’s  risk  mitigation  plans  in  preference  to  the  program’s  past  history 
or  current  status. 

•  The  PEO  should  make  risk  mitigation  funding  both  acceptable  and  available. 

The  second  category  is  budget  and  contracting  processes.  The  group  is  concerned  that 
EA/SD  are  not  synchronized  with  the  budget  process;  that  current  contractual  procedures  are 
not  sufficiently  flexible  (contracts  do  not  support  loosely  defined  efforts  and  do  not  support 
close  technical  working  relationships  between  government  and  contractors);  and  that  contrac¬ 
tors  are  reluctant  to  partner  with  competitors  (there  is  little  incentive  for  a  contractor  to  share 
information  or  products  with  companies  who  are  or  may  be  competitors).  Three  recommen¬ 
dations  were  made  in  this  category  with  respect  to  budget; 


CMU/SEI-2001-SR-005 


39 


•  OSD  should  develop  mechanisms  to  synchronize  funding  decisions  with  anchor  point 
milestones.  Date:  concurrent  with  the  release  of  5000.2R. 

•  OSD  should  support  the  Services  in  budgeting  C2  programs  in  the  aggregate  (at  the 
PEO  level)  to  enable  funding  of  spirals  on  a  realistic  and  timely  basis.  Date:  start  with 
the  ’02  Budget  Estimate  Submission  (BES). 

•  OSD  and  the  Services  should  promote  an  understanding  of  the  unique  funding  needs  of 
the  EA/SD  programs  by  the  financial  management  community  (to  avoid  cuts  of  funds 
essential  to  implementation  of  flexible  requirements  and  milestones).  Date:  concurrent 
with  release  of  5000.2R. 

Two  recommendations  were  made  in  this  category  with  respect  to  contracting: 

•  The  Defense  Contract  Management  Agency  should  develop  contracting  models  and 
suggest  incentives  that  match  EA/SD  models.  Date:  September  ’01. 

•  Program  managers  should  include  requirements  and  incentives  for  sharing  information 
and  products  in  all  appropriate  contracts. 

The  third  category  is  inappropriate  stakeholder  participation.  The  issues  in  this  category  in¬ 
clude  user  organizations  being  unwilling  to  offer  proactive  participants  (representatives  that 
are  not  empowered,  not  involved  enough,  unwilling  to  negotiate  requirements,  or  don’t  know 
what  is  needed);  the  test  community  being  unready  to  partner  in  the  development  process 
(don’t  know  how  to  apply  their  charter  in  an  EA/SD  environment  and  spiral  developers  no 
knowing  how  to  incorporate  testing);  IPT  participants  not  being  empowered  to  make  deci¬ 
sions  (including  testers,  users,  funders,  acquirers,  contractors  as  participants,  requirements 
and  risks  as  decisions).  Four  recommendations  were  made  in  this  category: 

•  OSD  should  update  IPT  language  to  include  a  broader  definition  of  “stakeholder”  for 
spiral  development  projects.  This  definition  should  include  oversight,  budget,  contrac- 
tor(s),  contracting,  and  test  organizations. 

•  Program  budgets  must  include  funding  for  participation  of  stakeholders  in  program 
activities. 

•  The  number  of  IPTs  that  require  end-user  support  must  be  limited  so  members  can  be 
empowered  and  knowledgeable. 

•  IPTs  should  be  given  the  authority  to  negotiate  requirements,  test  strategy,  and  budget 
allocations. 

The  fourth  category  is  lack  of  training  and  education.  The  group  was  concerned  that  there  is 
no  EA/SD  doctrine.  Such  a  doctrine  has  to  include  when  to  use  EA/SD,  terminology  that  is 
specific  to  the  processes,  and  training  manuals.  DoD  5(XX)  and  AFI 63-123,  which  mandate 
use  of  EA  and  SD  respectively,  are  not  doctrine  but  policy.  There  is  no  resource  that  provides 
a  clear  understanding  of  EA/SD — though  we  believe  the  Air  Force  manuals  are  a  start  at  this. 
In  addition,  there  is  a  lack  of  institutional  training  and  education  in  EA/SD.  Program  manag¬ 
ers  do  not  have  an  understanding  that  there  is  enough  flexibility  to  adopt  appropriate  process 
models  and  there  is  no  easily  accessible  source  or  proponent  for  EA/SD  to  support  program 
managers.  There  is  little  information  on  EA/SD  in  courses  provided  to  military  and  govern¬ 
ment  civilian  employees.  Specifically,  formal  educational  programs  fail  to  provide  education 
on  risk  patterns.  Five  recommendations  were  made  in  this  category: 
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.  FFRDCs  should  be  asked  to  develop  representative  risk  Pf ‘“n 
mitigation  precess  models  to  show  which  patterns^  b«tsmt^  toEA/SD.  (The  SEI 

risk  mitigation  database  is  a  good  Starting  point).  Date.  Feb  Ul. 

•  The  Services  should  validate  EA/SD  models  through  industry  and  government  pilot 
projects  as  part  of  the  Simulation  Based  Acquisition  Initiative.  Date;  September  0  . 

.  The  SEI  should  be  tasked  to  produce  a  case  study/examples  document.  Date:  Septem¬ 
ber  ‘01. 

.  The  OSD  should  require  EA/SD  curricula  be  incorporated  in  Professional  Military 
Education  (PME)  and  Defense  Acquisition  University  (DAU)  for  stak^olders  as  well 
"won^Lrionals.  Suggesred  components  are  listed  below.  Datet  February 

‘02. 

.  The  OSD  should  provide  a  contract  vehicle  for  EA/SD  facilitators  artd  mentors  to  sup- 
port  early  adopters  of  EA/SD. 

In  thinking  about  training,  the  group  came  up  with  some  recommendaUons  as  to  what  should 
be  covered  in  a  curriculum.  The  group  felt  that  the  following  informatton  should  be  pmvtded 

to  PEOs: 


•  invariants/variants 

•  case  histories  (including  EMD  and  Deployments  phases) 

•  risk  management  (doing  the  hard  things  first) 

The  group  felt  that  potential  PMs  should  be  sent  to  DAKPA  or  NASA  in  order  to  participate 
in  transition  of  technologies  key  to  their  programs. 


The  course  for  end-users  should  cover 

•  the  role  of  negotiation  and  flexible  requirements 

•  how  the  ORD  affects  program  decisions,  including  upgrade  plans 

•  what  to  expect/demand  from  executable  prototypes 

•  demonstration  of  program  risk  patterns 

.  expectations  from  model  based  software  engineering  (i.e.,  MBASE) 


Summary 

It  is  the  group’s  fervent  hope,  if  not  actual  belief,  that  foUowing  our  recommendations  in  the 

areas  of 

•  risk  aversion 

•  budget  and  contracting  processes 

•  inappropriate  stakeholder  participation 

•  lack  of  training  and  education 


will  help  achieve  our  vision  for  acquisiUon  in  2010.  That  is,  one  where  projects  always  fol¬ 
low  appropriate  process  nMdels,  base  their  behavior  on  a  full  undeistandmg  of  the  risks,  an 

acquire  needed  capabilities  sooner  and  at  lower  cost. 
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5.5  Work  Group  “Operationalization”:  Practical  Steps 
Necessary  to  Operationalize  Spiral  Development 
and  Evolutionary  Acquisition 

Panel:  Joseph  Ferrara  (co-chair  Georgetown  University),  Eileen  Forrester 

(co-chair,  SEI),  Thomas  Bono  (MITRE),  Mike  Bloom  (MITRE),  Ed 
Cameron  (US  Army),  Peter  Hantos  (Xerox),  Cheryl  Jones  (US  Army 
TACOM),  Tony  Jordano  (SAIC),  Larry  McKee  (MITRE),  John  Miller 
(TRW),  Calvin  Young  (Quantum  Research  International) 

Work  Group  5  considered  what  steps  the  Department  of  Defense  (DoD)  should  take  to  opera¬ 
tionalize  and  ultimately  institutionalize  evolutionary  acquisition  and  spiral  development 
(EA/SD).  Its  objective  was  to  be  practical  and  realistic,  and  wherever  possible,  avoid  recom¬ 
mendations  that  aimed  for  the  quick  fix  and  ignored  the  practical  realities  of  change  man¬ 
agement  in  large,  complex  organizations. 

Guiding  Principies 

The  group  discussion  was  guided  by  a  few  broad  principles.  First,  the  group  did  not  assume 
that  EA  and  SD  were  necessarily  always  the  most  appropriate  strategy  and  tool.  Rather,  as 
the  DoD’s  new  5000  policy  documents  say,  the  group’s  basic  assumption  was  that  the  success 
of  a  DoD  program  is  fundamentally  dependent  on  crafting  an  acquisition  strategy  that  is  re¬ 
sponsive  to  that  program’s  unique  blend  of  characteristics.  In  other  words,  successful  opera¬ 
tionalization  of  EA/SD  requires  support  for  selecting  the  right  life-cycle  model  for  the  cir¬ 
cumstances;  EA/SD  may  be  it,  and  we  should  ensure  that  EA  is  selected  when  it  is  the  best 
match.  EA/SD  is  not  a  silver  bullet. 

A  second  guiding  principle  was  the  notion  that  not  all  stakeholders  are  created  equal.  That  is, 
for  any  given  change  management  initiative  (in  this  case,  EA/SD),  one  can  define  a  set  of 
critical  stakeholders  whose  early  adoption  is  vital  to  ultimate  success  because  it  attracts  other 
followers  and  brings  them  along  as  the  organization  moves  toward  institutionalization.  We 
noted  that  EA/SD  has  a  large  number  of  potential  stakeholders  and  we  endeavored  to  focus 
on  the  most  important  for  operationalization  of  EA. 

Third,  the  group  was  struck  by  the  need  for  the  champions  of  EA/SD  to  tell  a  consistent  story. 
As  was  noted  several  times  during  the  workshop,  DoD  spokespersons  have  not  always  been 
clear  in  elucidating  the  basic  building  blocks  of  the  EA/SD  approach.  (In  part,  this  is  due  to 
the  messiness  of  the  policy  development  process  and  the  inevitable  need  to  make  compro¬ 
mises  along  the  way.  But  now  that  the  new  5000  policies  have  been  formally  signed  out,  it  is 
very  important  that  DoD  begin  to  speak  with  one  voice.)  We  recognize  the  need  for  consis¬ 
tent  communication  from  the  various  sponsors  and  champions  of  a  change  to  appropriate  use 
ofEA. 
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Next,  the  group  felt  the  proper  scope  of  its  task  was  to  focus  not  on  individual  organizations 
(say,  the  Army,  or  the  Naval  Air  Systems  Command),  but  rather  on  the  broader  defense  ac¬ 
quisition  community.  Operationalization  is  a  community-wide  enterprise  that  will  require 
buy-in,  collaboration,  and  cooperation  from  a  diverse  set  of  contributors. 

A  final  guiding  principle  was  the  recognition  that  institutionalization  of  EA/SD  will  require  a 
whole  product  -  not  just  piece  parts  -  and,  moreover,  that  this  product  will  require  systematic 
leadership  attention  and  interdisciplinary  collaboration. 

Summary  of  Discussion 

To  make  the  most  of  the  brief  time  allotted,  the  group  quickly  identified  the  key  issues  its 
members  felt  were  most  important  to  success.  This  had  two  advantages  -  it  gave  the  group 
an  opportunity  at  the  outset  to  brainstorm  a  number  of  possible  solutions,  and  then  to  organ¬ 
ize  these  solutions  into  coherent  sets  of  transition  mechanisms.  Next,  the  group  identified- 
and  prioritized-the  stakeholders  important  to  institutionalization.  After  this,  the  group 
mapped  the  various  transition  mechanisms  along  a  path  representing  the  phases  of  organiza¬ 
tional  commitment,  stretching  from  initial  contact  to  full  institutionalization  [Conner  83]. 
Finally,  the  group  made  a  top-level  recommendation  about  the  need  for  a  full-time  steward  to 
guide  transition  and  adoption. 

It  is  important  to  note  two  things  the  working  group  did  not  do.  First,  it  avoided  a  recitation 
of  familiar  litanies.  The  sense  of  the  group  was  that  litanies,’  by  their  very  nature,  tend  to  be 
trite  and  to  induce  a  certain  numbness  in  the  listener,  and  so  the  group  avoided  them  (with 
some,  but  not  total,  success).  Second,  the  working  group  did  not  attempt  to  draft  a  compre¬ 
hensive  plan  for  operationalization.  Such  a  goal-even  if  the  group  had  adopted  it  -  would 
have  been  precluded  by  the  severe  time  constraints  in  any  event,  but  the  group  also  recog¬ 
nized  that  its  purpose  was  to  point  a  way  for  others  to  follow,  not  to  wnte  the  detailed  blue- 

print. 

stakeholder  Communities 

Who  are  the  relevant  stakeholder  communities?  The  group  identified  three  main  categories: 
(1)  Program  Management  and  Delivery,  (2)  Program  Support,  and  (3)  Policy  and  Oversight. 

See  Table  3. 


'A  good  example  of  a  litany  is  the  oft-heard  argument  that  “if  only  Congress  would  not 

micromanage,”  then  all  things  would  be  better.  This  may  be  true  but  it  is  also  utterly  impractical 
and  unrealistic. 
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Table  3:  EA/SD  Stakeholder  Communities 


Popuiation 

Comments 

Program 
Management 
and  Delivery 

Program  Managers  (in  government 
and  industry) 

Members  of  the  Program  Office  staff 
Contracting  Officers 

Program  Executive  Officers 

Designated  Acquisition  Commanders 
Chief  Engineers  and  Architects 

This  is  the  group  most 
directly  responsible  for 
program  success;  for 
meeting  the  bottom-line 
goals  of  cost,  schedule, 
and  performance. 

Program 
Support . 

Logisticians 

Trainers  and  Educators 

Operators 

Requirements  Writers 

Testers 

Scientists  and  Technologists 

This  group  supports 
program  management  in 
the  broadest  sense: 
everything  from  writing  the 
requirements  from  which 
programs  originally  flow  to 
providing  training  and 
education. 

Policy  and 
Oversight 

Congress 

Acquisition  Executives  and  Milestone 
Decision  Authorities 

Overarching  IPTs 

Acquisition  Reformers 

Auditors 

Budget  Officers  and  Programmers 

This  group  writes  policy 
and  provides  oversight; 
both  of  individual 
programs  on  a  case-by- 
case  basis  and  of  policy 
compliance  in  general. 

After  discussing  the  roles  and  responsibilities  of  these  various  stakeholders,  the  work  group 
decided  that  the  Program  Management  and  Delivery  community  was  by  far  the  most  impor¬ 
tant  to  focus  on  in  developing  a  plan  for  organizational  commitment  and  institutionalization. 
This  decision  bears  some  elaboration.  First,  this  choice  does  not  imply  that  the  other  stake¬ 
holder  communities  are  somehow  unimportant.  They  are  very  important;  however,  we  can¬ 
not  do  everything  at  once.  Because  fundamentally,  EA/SD  is  about  program  strategies,  it 
makes  sense  to  focus  on  the  conununity  most  directly  responsible  for  program  success.  Sec¬ 
ond,  in  a  sense,  the  policy  and  oversight  conununity  is  already  committed  to  EA/SD,  having 
just  issued  the  new  5000  policies.  And  so  constructing  an  elaborate  program  to  convince  this 
community  of  the  merits  of  EA/SD  would  seem  to  be  superfluous,  a  classic  case  of  preaching 
to  the  choir.  Third  and  most  importantly,  this  choice  reflects  the  work  group’s  desire  to  be 
practical  and  realistic.  As  one  workshop  participant  put  it,  to  be  truly  successful,  EA/SD 
“must  get  into  the  very  DNA  of  program  managers.”  If  those  responsible  for  program  man¬ 
agement  and  delivery  don’t  adopt  EA/SD,  it  doesn’t  matter  if  the  others  do — 
operationalization  will  fail. 

A  Phased  Approach 

The  core  of  the  work  group’s  strategy  for  operationalization  is  a  phased  implementation  ap¬ 
proach  in  which  adopters  of  the  EA/SD  policy  are  addressed  in  a  meaningful,  practical  se- 
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quence.  This  approach  offers  several  advantages.  It  flows  from  the  finding  that  certain 
stakeholder  communities  are  more  critical  than  others  to  ultimate  success  and  that  such  early 
adopters  can  play  a  catalytic  role  in  organizational  change.  In  addition,  a  phased  approach 
recognizes  that  people  accept  change  in  fairly  predictable  patterns.  Rarely  do  people  (much 
less  whole  organizations)  fall  head  over  heels  in  love  with  a  new  technology  and  realize  in¬ 
stantaneously  how  to  put  it  into  practice.  Typically,  it  is  not  love  at  first  sight  but  rather  more 
akin  to  a  gradual  courtship  where  people  learn  more  about  the  new  initiative,  overcome  their 
fears  and  initial  resistance,  and  begin  to  understand  its  real  benefits. 

In  this  vein,  the  work  group  based  its  approach  on  research  from  the  training  and  develop¬ 
ment  literature  that  posits  a  phased  commitment  to  organizational  change  over  time,  a  com¬ 
mitment  that  grows  in  intensity  as  the  organization  moves  from  basic  awareness  to  actual  use 
and  demonstration  of  the  new  concepts.  A  graphic  representation  of  this  phased  commitment 
path,  with  relevant  mechanisms  for  adopting  EA/SD,  is  depicted  in  Figure  9. 


Figure  9:  How  Organizations  Commit  to  Change 
Table  4  below  briefly  elaborates  on  each  of  the  stages  of  organizational  change  process. 
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Table  4:  Stages  of  the  Organizational  Change  Process _ _ 

Stage  Objectives  Example  l\/lechanlsms 

Contact  and  Awareness 

Intended  adopters  recognize  the  Elevator  speech 
name  Magazine  articles 


Basic  grasp  of  what  the  new  tech-  PAQs,  iiiustrative  success  stories,  points 

nology  is  and  what  benefits  it  of-  gf  contact,  etc. 

Standard  explanatory  briefings  for  delivery  at  profes¬ 


sional  conferences 


Understanding 

Deeper  comprehension  of  new 
initiative  and  its  essential  tenets 
and  techniques 

More  investment  by  adopters  of 
time,  effort,  or  other  resources  that 
will  lead  to  a  decision  to  try  it  out 


One-day  seminars 
Technical  briefings 

More  detailed  case  studies  and  success  stories 
Road  shows  (including  program  managers) 

Updating  existing  course  content  at  DoD  school 
houses  to  reflect  new  initiative 


Trial  Use 

Gradual  organizational  experimen¬ 
tation  with  new  approach  to  dem¬ 
onstrate  its  value  and  learn  les¬ 
sons  for  future  improvement 

Enough  information  to  make  an 

adoption  decision 

Word  of  mouth  on  results  or  other 

references 


Carefully  selected  lead  programs 

Commitment  to  support  lead  programs  and  document 

lessons  learned 

Establishment  of  a  senior  group  dedicated  to  support¬ 
ing  and  nurturing  lead  programs 
Development  of  longer,  more  detailed  course  offerings 

Documentation  of  barriers  to  adoption  and  strategies 
to  overcome  these  barriers 


Adoption 

Transition  beyond  trial  use  to  a  Establishment  of  a  robust  set  of  organizational  incen- 

more  mature  organizational  adop-  tives,  rewards,  and  consequences 

tion  of  the  new  approach,  including  Refining  policy  and  procedural  guidance  to  reflect 

policy  refinement  and  comprehen-  lessons  learned  through  trial  use 

sive  education  and  training  Development  of  a  full  suite  of  in-process  aids  and 

Information  on  costs  and  benefits  tools,  including  sample  implementation  plans,  guide- 

plus  other  lessons  In  trial  use  books,  team  coaching  services,  etc. 

Preparation  for  rollout  to  common  Leadership  attention  to  and  resolution  of  lingering 
practice  adoption  issues 


Institutionalization 

Full  inculcation  of  the  new  ap-  A  fully  realized  educational  curriculum 

proach  Into  the  culture  and  norms  systematic  program  of  orientation  and  training  for 

of  the  organization  nevv  employees  as  they  enter  the  organization 

Refreshment  and  upkeep  Stability  of  leadership  commitment 

Reinforcement  and  adaptation  Continuous  improvement  of  adoption  artifacts 
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An  Overarching  Recommendation 

How  exactly  is  DoD  to  initiate  the  actions  described  above?  Who  will  take  the  lead  and  keep 
things  on  track?  Who  will  ensure  that  the  necessary  actions-sniall  and  large-are  tasked  and 
accomplished?  The  work  group  felt  that  this  was  the  most  severe  and  pressing  challenge  fac¬ 
ing  advocates  of  EA/SD.  There  are  government  warehouses  filled  to  bursting  with  new  pol¬ 
icy  documents,  sparkling  briefings  and  guidebooks,  and  transition  plans  of  all  make  and 
manner.  But  all  of  these  products  will  not  ensure  success  without  consistent,  systematic  lead¬ 
ership.  A  major  concern  of  the  work  group  was  that  the  nature  of  DoD  bureaucracy,  particu¬ 
larly  the  policy  and  oversight  stakeholder  community,  is  to  declare  victory  and  move  on  to 
new  challenges  after  a  new  policy  is  published.  In  this  scenario,  implementation  and  institu¬ 
tionalization  too  often  become  mere  afterthoughts.  But  of  course  the  publication  of  a  new 
policy  is  really  just  the  beginning. 

This  is  the  animating  principle  that  runs  through  the  work  group  discussion  laid  out  above. 

The  publication  of  the  new  5000-far  from  the  end  of  a  long  road  of  policy  development-is  in 
fact  only  the  beginning  of  the  organizational  change  process.  As  we  have  argued,  this  proc¬ 
ess  must  be  planned,  managed,  and  measured  and  must  focus  on  first  things  first.  But  some¬ 
one  must  move  things  along.  Who  should  that  be? 

It  was  the  work  group’s  strong  recommendation  that 

•  DoD  appoint  and  fund  an  organization  to  serve  as  steward  for  ensuring  the  transition 
and  adoption  of  EA/SD. 

It  is  desirable  that  this  organization  be  well  positioned  to  deal  with  the  full  range  of  stake¬ 
holder  communities,  including  government,  industry,  and  academia.  Moreover,  it  should  be 
an  organization  that  possesses  demonstrable  expertise  in  transitioning  new  technologies  and 
practices  to  widespread  use  and  adoption.  It  must  also  be  perceived  as  being  relatively  neutral 
and  objective.  Consideration  of  these  desired  characteristics  very  quickly  focused  the  group 
on  Federally  Funded  Research  and  Development  Centers  (FFRDCs).  It  is  part  and  parcel  of 
the  FFRDC  mission  to  assist  the  government  sponsor  in  building  bridges  to  industry  and  aca¬ 
demia.  In  particular,  the  work  group  strongly  believed  that  the  SEI,  because  of  its  FFRDC 
status  and  even  more  importantly  because  of  its  technology  transition  mission,  is  a  logical 
candidate  to  serve  as  the  EA/SD  steward. 

This  recommendation  has  several  advantages.  First  and  most  importantly,  it  ensures  that 
rollout  and  implementation  of  EA/SD  does  not  become  a  bureaucratic  afterthought.  As  noted 
above,  the  publication  of  the  new  5000  marks  the  beginning,  not  the  end,  of  a  long  process. 
Second,  appointing  a  steward  is  practical  and  realistic,  and  helps  government  leaders  do  their 
jobs.  The  leadership  of  the  DoD  acquisition  community  is  busy  and  often  spread  far  too  thin. 
Having  a  steward  to  promote  policy  transition  and  adoption  helps  everybody.  Third,  a  stew¬ 
ard  can  reach  out  to  all  affected  communities  and  map  the  complex  interactions  necessary  for 
successful  adoption.  A  clear  finding  of  the  workshop  was  that  institutionalizing  EA/SD 
would  require  numerous  actions  across  a  range  of  communities  and  disciplines. 
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To  make  this  happen,  the  work  group  recommends  that  the  SEI  meet  with  its  DoD  sponsors 
as  soon  as  possible  to  brief  the  results  of  the  EA/SD  workshop  and  to  recommend  the  ap¬ 
pointment  of  SEI  as  EA/SD  steward.  Key  offices  to  be  consulted  include  the  Deputy  Under 
Secretary  for  Science  and  Technology  (sponsor  of  the  SEI),  the  Director  of  Acquisition  Re¬ 
sources  and  Analysis,  the  Deputy  Under  Secretary  of  Defense  for  Acquisition  Reform,  and 
the  Service  Acquisition  Executives. 


48 


CMU/SEI-2001-SR-005 


6  Conclusions 


EvoluUonary  acquisitton  (EA)  and  spiral  davelopmen,  (SD)  are  ,he  c^n.  eM  to  de¬ 
fine  an  acquisition  process  that  consistently  attains  that  uncomfortably  small  apex  on 
p;a:”d1^Lg  low  cost,  rapid  delivery,  and  high  quality.  Ouanling  the  apex  are  arrayed  the 

forces  of  nature,  as  shown  in  Figure  10. 


Figure  10:  The  Forces  of  Nature  Guard  the  Apex  (^Process  Achievement 

Them  was  no  dissent  at  either  the  February  or  the  September  workshop  to  the  notion  ^ 
EA/SD  can  help  attain  the  apex  more  often  than  other  process  mode  s. 
erally  enthusiastic  about  the  prospects.  Nevertheless,  the  presentations  and  (especially) 
lecommendations  can  he  read  as  leading  to  this  cynical  mterpretatiom 
We  don’t  understand  EA  and  SD 
and  we  aren’t  convinced  they  work, 
but  we  want  them  anyway  so  here’s  how  to 
improve  them,  adapt  the  DoD,  make  teams  work,  and 

ensure  their  use. 
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Let  us  consider  each  phrase  in  turn,  examining  its  roots  and  the  reasons  for  hope. 


We  don’t  understand  EA  and  SD 

The  notion  that  EA  and  SD  are  not  well  understood  could  be  garnered  from  the  fact  that  their 
definitions  were  one  of  the  themes  of  the  February  presentations  and  the  subject  of  work 
groups  at  both  workshops.  In  fact,  the  work  group  sessions  were  intended  to  sharpen  the 
definitions  rather  than  to  create  new  ones.  The  February  work  group  did  formulate  four  rec¬ 
ommendations  about  further  sharpening  the  definition,  but  the  September  group  only  formu¬ 
lated  recommendations  calling  for  further  promulgation  of  the  definitions. 

Largely,  the  lack  of  September  recommendations  reflects  the  existence  of  definitions  that 
came  into  being  between  the  workshops.  The  definitions  of  evolutionary  acquisition  are 
given  in  Section  3.1.  Spiral  development  is  defined  in  a  recent  paper  [Boehm  00,  Boehm  01] 

as  follows: 

The  spiral  development  model  is  a  risfe-driven  process  model  generator.  It  is  used  to 
guide  multi-stakeholder  concurrent  engineering  of  software-intensive  systems.  It  has 
two  main  distinguishing  features.  One  is  a  cyclic  approach  for  incrementally  growing  a 
system’s  degree  of  definition  and  implementation  while  decreasing  its  degree  of  risk. 

The  other  is  a  set  of  anchor  point  milestones  for  ensuring  stakeholder  commitment  to 
feasible  and  mutually  satisfactory  system  solutions. 

The  papers  go  on  to  sharpen  the  definitions  by  defining  the  highlighted  terms  and  describing 
“invariant”  properties  of  spiral  processes  together  with  allowable  “variants  for  each. 

One  mistake  common  among  development  practitioners  is  to  confuse  spiral  development 
with  any  other  method  where  the  product  is  developed  in  a  cyclic  sequence  of  efforts  rather 
than  one  single  waterfall.  Usually  such  efforts  are  conducted  without  risk  management  re¬ 
views  and  re-direction  of  the  project  at  the  beginning  of  each  cycle.  One  example  of  this  sort 
of  confusion  is  the  phrase  in  DoDI  5000.2  where  it  calls  for  an  “iterative  spiral  approach”  for 
development.  Does  this  mean  that  there  are  spiral  approaches  that  are  not  iterative?  Or  does 
it  try  to  clarify  the  meaning  of  “spiral  approach”  by  equating  it  with  an  iterative  approach? 
There  is  no  way  to  tell.  The  first  is  slightly  incorrect  insomuch  as  a  first  iteration  risk  analy¬ 
sis  may  determine  that  only  a  single  iteration  is  needed.  The  second  is  more  incorrect  m  that 
there  are  many  iterative  process  models  that  are  not  spiral  processes. 

Other  cases  where  iterative  was  equated  with  spiral  occurred  in  the  February  presentations, 
but  not  in  those  of  September.  These  instances  of  confusion  probably  reflect  a  deep-seated 
misunderstanding  which  will  require  much  education  to  reverse.  (Perhaps  it  would  be  useful 
to  create  a  new  name  for  the  risk-based  spiral  method  rather  than  try  to  re-educate  everyone 
on  the  meaning  of  “spiral  development.”) 

and  we  aren’t  convinced  EA  and  SD  work 

The  work  groups  suggest  an  uncertainty  with  the  validity  of  SD  and  EA  by  asking  for  various 
kinds  of  demonstrations.  These  were  the  subject  of  at  least  10  recommendations  by  6  differ- 
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ent  work  groups.  Typically  these  recommendations  called  for  case  studies,  expenmental  vali¬ 
dation,  pilot  projects,  and  retum-on-investment  or  business  case  analyses. 

In  point  of  fact,  workshop  participants  were  themselves  convinced  as  to  the  merits  of  SD  and 
EA  The  spate  of  recommendations  was  intended  to  provide  the  sort  of  concrete  evidence  that 
is  the  best  argument  to  present  to  skeptical  and  unwilling  potential  adopters.  Without  a 
strong  body  of  evidence,  no  amount  of  regulations  and  documentation  can  break  a  large, 
long-lived  system  free  from  the  clutches  of  time-blessed  procedures. 

Little  has  been  accomplished  on  the  building  of  convincing  evidence  for  EA  and  SD  since  the 
time  of  the  first  workshop  in  February.  It  is  possible  this  goal  will  be  accomplished  by  the 
recently  funded  Center  for  Empirically  Based  Software  Engineering  [CeBASE  01]  or  some 
other  group  utilizing  the  resources  of  the  SEI  Software  Engineering  Information  Repository 

[SEIROl]. 

but  here’s  how  to  improve  EA  and  SD 

Although  participants  generally  agreed  that  EA  and  SD  were  valuable  techniques,  they  had 
suggestions  for  improvement  in  several  directions.  On  the  “people”  dimension,  these  sugges¬ 
tions  ranged  from  working  to  understand  the  human  behavioral  and  cultural  aspect  down  to 
the  practical  task  of  devising  methods  to  encourage  the  identification  and  revelation  of  nsks. 
On  the  “process”  dimension,  suggestions  included  development  of  tools,  testing  practices, 
and  project  assessment  methodologies.  One  particular  suggestion  called  for  a  catalog  of  nsk 
patterns  and  the  corresponding  appropriate  process  pattern.  At  least  two  of  the  presentations 
described  such  catalogs  in  commercial  use  (Kitaoka  in  February  and  Jordano  in  September). 
These  suggestions  were  aU  aimed  at  making  EA  and  SD  easier  to  use  and  thus  more  attractive 

for  future  projects. 

here’s  how  to  adapt  DoD  to  EA  and  SD 

Given  that  EA  and  SD  are  seen  as  attractive  strategies,  and  given  the  fact  that  no  other  strate¬ 
gies  have  been  particularly  successful,  workshop  participants  determined  that  the  DoD  should 
encourage  these  policies.  However,  many  factors  in  existing  DoD  policies,  practices,  and  in¬ 
grained  culture  were  seen  as  deterrents  to  the  successful  adoption  of  EA  and  SD.  It  is  no 
wonder  then,  that  the  work  groups  proposed  a  number  of  changes  for  the  DoD.  These 
changes  have  been  detailed  in  the  work  group  reports  and  summarized  under  the  categones: 

•  Improve  Contract  Models 

•  Revise  Funding  Approaches 

•  Adapt  Acquisition  Policies 

Noteworthy  recommendations  appeared  in  all  these  areas.  In  contracting,  one  recommenda¬ 
tion  sought  to  dispel  the  parochial  atmosphere; 

•  Include  requirements  and  incentives  for  sharing  information  and  products  in  all 
appropriate  contracts. 
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In  the  policy  area,  recommendations  called  for  coordination  of  personnel  changes  with  spiral 
cycles  and  focusing  program  oversight  on  risk  management  plans  instead  of  the  amorphous 
notion  of  status  versus  plan.  In  the  funding  area,  the  September  “Barriers”  work  group  ex¬ 
pressed  the  opinion  that  there  were  no  policy  barriers  to  writing  contracts  to  implement  EA 
and  SD.  Nonetheless,  a  number  of  far-reaching' recommendations  concerning  funding  were 
made  by  various  groups; 

•  Work  with  Congress  to  improve  the  funding  model  for  DoD  projects. 

•  Synchronize  funding  decisions  with  anchor  point  milestones. 

•  Budget  C2  programs  in  the  aggregate  (at  the  PEO  level). 

•  Help  financial  managers  understand  the  unique  funding  needs  of  EA/SD. 

•  Make  risk  niitigation  funding  both  acceptable  and  available. 

here’s  how  to  make  teams  work 

Teamwork  is  a  military  hallmark.  Without  teamwork,  battles  are  lost  before  begun.  Consider 
the  teamwork  displayed  by  the  movement  of  176,000  troops  on  4100  ships  to  accomplish  the 
Normandy  beachhead  on  D-Day.  Despite  this  glorious  history,  acquisition  more  often  seems 
an  adversarial  skirmish  than  a  team  working  to  accomplish  a  task.  Of  course,  contractors 
have  adversaries;  this  is  guaranteed  by  the  system  of  competitive  bidding.  The  adversarial 
attitude  has,  too  often,  been  carried  forward  into  the  execution  of  the  contract,  once  awarded. 
Contractors  are  loath  to  share  information  with  other  contractors  or  even  with  the  customer 
whose  contract  explicitly  states  a  right  to  information. 

Many  efforts  have  been  made  to  forestall  adversarial  interactions,  notably  “integrated  product 
teams”  formed  from  stakeholders  including  the  acquirers,  the  contractors,  the  users,  and 
many  more.  This  concept  has  even  been  adopted  as  part  of  the  Spiral  Development  Model. 
Nonetheless,  it  does  not  always  work  to  the  full  satisfaction  of  all.  Observing  this,  the  work 
groups  had  many  suggestions  about  teamwork  and  teamwork  was  a  minor  theme  of  the  Feb¬ 
ruary  presentations  and  a  major  theme  of  the  September  presentations. 

Many  specific  recommendations  were  expressed.  Among  them,  the  most  vital  is  that 

IPTs  should  be  given  the  authority  to  negotiate  requirements,  test  strategy,  and  budget 
allocations 

Other  recommendations  described  steps  to  take  to  ensure  a  representative  and  engaged  team. 
The  plan  for  the  team  should  include  team-building  efforts.  If  the  team  is  distributed  over  a 
number  of  locations,  it  should  have  collaboration  tools  and  various  other  recommended  steps 
should  be  taken  to  maximize  success. 

and  here’s  how  to  ensure  use  of  EA  and  SD 

Given  the  workshop  enthusiasm  for  the  spiral  development  method,  it  is  not  surprising  that 
participants  devoted  attention  to  how  to  foster  the  use  and  growth  of  the  method  for  military 
acquisition.  Changing  existing  habits  is  always  difficult.  It  does  not  help  that  the  acquisition 
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process  involves  miliury  personnel  who  can  be  ••ordered”  ,o  do  things  in  a  given  way  a^ 
also  contractors  who  can  be  contractually  ••bound"  to  do  things  in  that  sai*  way.  Neither 
stricture  matters  when  people  go  off  to  do  their  actual  work  and  revert  to  hoary-aged  habits. 

The  emphasis  on  promulgation  of  EA  and  SD  was  evident  in  the  workshops  in  that  it  was  a 
presentation’s  theme  in  September,  two  categories  of  recommendation  in  Fetaary  (Promote 
L  and  Educate  About  SD).  and  a  recommendation  category  m  September.  However,  there 
were  no  really  innovative  ideas  about  how  to  accomplish  the  change,  other  than  the  work  o 
the  “Operattonalization”  group  in  September.  The  majority  of  the  recommendauons  called 
for  wriUng  guidebooks,  case  studies,  or  other  materials-all  with  no  real  incentives  for  peo¬ 
ple  to  actually  read  the  results.  A  second,  more  promising,  collection  of  recommendanons 
Lked  at  the  longer  term  and  suggested  emphasis  on  SD  in  various  con^s  that  ac^uon 
personnel  may  encounter,  including  universities.  Professional  Military  Education^ME), 
Defense  Acquisition  University  (DAU),  and  the  Project  Management  Institute  (PMl). 

Other  recommendations  were  to 

•  Deliberately  build  an  SDM  community. 

.  Provide  a  contract  vehicle  for  EA/SD  facilitators  and  mentors  to  support  early 
adopters  of  EA/SD. 

•  Develop  a  cadre  of  industry  and  government  people  knowledgeable  in  the 
pragmatic  application  of  EA  and  SD  who  are  readily  available  to  program  of¬ 
fices  and  contractors. 

.  Pay  greater  attention  to  education  and  selection  of  integration  practitioners. 

The  Real  Situation 

This  discussion  was  prompted  by  a  fairly  cynical  opinion,  which  can  now  be  interpolated 
with  a  more  realistic  evaluation: 

We  don ’t  understand  EA  and  SD 

Although  not  widely  understood,  the  definitions  are  well  established. 
and  we  aren ’t  convinced  they  work, 

Real  successes  are  documented,  but  systematic  study  is  indeed  needed. 
but  we  want  them  anyway  so  here’s  how  to 
improve  them,  adapt  the  DoD,  make  teams  work. 

The  workshops  produced  valuable  suggestions  in  all  three  of  these  areas. 

and  ensure  their  use. 

Additional  documents  may  help,  but  education  is  the  only  real  solution. 

EA  and  SD  provide  good  process  models  for  acquisition  by  the  military  and  development  by 
contractors.  These  tasks  are  notoriously  prone  to  descent  on  the  slopes  of  Figure  10  to  budget 
overruns,  dilatory  delivery,  and  ill-designed,  poorly  executed  products.  Thus  EA  and  SD  are 
worth  pursuing  for  the  promise  shown  by  successful  expenences  reaching  back,  in  some 
cases,  over  one  or  two  decades. 
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As  part  of  analyzing  the  recommendations  for  the  above  summary,  one  set  of  recommenda¬ 
tions  stood  apart.  These  were  the  recommendations  that  a  single  focal  point  for  EA/SD  im¬ 
provement/adoption  be  established.  These  recommendations  called  variously  for  an  IPT,  an 
office  within  SIS,  or  an  outside  organization.  Whichever  is  established,  its  purpose  should  be 
to  shepherd  the  recommendations  of  the  workshop  and  other  efforts  at  improving  EA,  SD, 
and  their  use  in  acquiring  military  systems.  Such  an  office  is  imperative  to  avoid  duplicate 
efforts  and  neglected  opportunities.  This  office  will  serve  as  the  single  point  of  information 
on  EA/SD  acquisition  policy,  practice,  and  guidance.  Project  managers  will  come  to  see  this 
office  as  the  source  of  training  materials  they  have  used  and  the  source  of  useful  suggestions 
and  tools  they  can  apply  to  the  current  tasks. 

These  two  spiral  development  workshops  have  generated  interest  in  EA  and  SD,  widened 
knowledge  of  their  merits  and  applicability,  and  brought  numerous  suggestions  for  improve¬ 
ment  of  all  these.  As  a  result,  and  with  the  aid  of  the  focal  office  recommended  by  the  work¬ 
shops,  it  is  possible  to  foresee  the  general  maturation  of  EA  and  SD  leading  to  a  time  when 
projects  are  routinely  economical,  timely,  and  useful. 

Returning  to  the  triple  apex  problem  of  attaining  early,  cheap,  good,  we  see  that,  in  a  sense, 
spiral  development  guarantees  two  of  these.  At  the  start  of  each  iteration,  risk  analysis  de¬ 
termines  what  is  to  be  done  and  how  long  to  take.  Budget  considerations  are  part  of  this,  so 
nothing  will  be  attempted  which  cannot  be  paid  for.  Given  these  constraints  it  may  not  be 
possible  to  meet  all  the  requirements.  That  is,  choosing  spiral  development  is  tantamount  to 
accepting  that  full  functionality  may  not  be  delivered  by  a  given  contract.  No  contract  is  thus 
a  fixed  price  contract.  The  risk  management  team — composed  of  members  from  the  con¬ 
tracting,  acquiring  and  using  communities — ^is  responsible  for  delivery  of  as  much  as  possible 
within  the  time  and  budget  allotted.  Thus  with  spiral  development  within  evolutionary  acqui¬ 
sition,  the  apex  of  success  can  always  be  attained! 
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Appendix:  Recommendations 


This  table  lists  the  recommendations  made  by  the  work  groups  sorted  according  to  class.  For 
the  full  text  of  the  recommendations,  see  the  report  above  or  the  version  on  the  Web  site: 

<http://www.sei.cmu.edu/cbs/spiral2000/september/recommendations-sept00.xls 


Work  Gp 

Class 

Agent 

Recommendation 

Due  date 

Definition 

^dapt 

SIS  Office 

dentify  focal  point  in  SIS  Office  for  EA  i 

Apr ’01 

Operationali¬ 

zation 

Adapt 

DoD 

1 

Appoint  and  fund  an  organization  to  serve  as  steward  ' 
or  ensuring  the  transition  and  adoption  of  EA/SD.  i 

with  release 
of5000.2R 

Mapping 

Adapt 

Ferguson, 

etal. 

Convene  an  IPT  to  further  the  incorporation  of  SD 
within  EA. 

Mapping 

Adapt 

PM 

Plan  each  EA/SD  project  so  consecutive  spiral  cycles 
converge  toward  a  final  goal . . .  use  anchor  pioints  or 
stable  intermediate  points. 

Mapping 

Adapt 

SIS  Office 

Depict  EA  so  as  to  reveal  the  feedback  possibilities 
instead  of  making  it  appear  as  a  rigid  linear  approach. 

Mapping 

Adapt 

PEO 

In  applying  ENSD  distinguish  between  Information 
Systems  and  Software  Intensive  Systems. 

Barriers  - 
Contracting 

Con¬ 

tract 

DCMA 

Develop  contracting  models  and  suggest  incentives 
that  match  E/VSD  models. 

Sep ’01 

Barriers  - 
Contracting 

Con¬ 
tract  , 

PM 

Include  requirements  and  incentives  for  sharing  in¬ 
formation  and  products  in  all  appropriate  contracts. 

Mapping 

Con¬ 

tract 

PM 

Avoid  SD  for  the  manufacture  of  any  lot  in  programs 
with  production  phases. 

Barriers  -  | 

Budget 

Fund 

OSD  n 

Develop  mechanisms  to  synchronize  funding  deci¬ 
sions  with  anchor  point  milestones. 

with  release 
of5000.2R 

Barriers  - 
Budget 

Fund 

OSD  and 
the  Ser¬ 
vices 

Budget  C2  programs  in  the  aggregate  (at  the  PEO 
level). 

start  with  ’02 
BES 

Barriers  - 
Budget 

Fund 

OSD  and 
the  Ser¬ 
vices 

Promote  understanding  of  unique  funding  needs  of 
the  E/V/SD  programs  by  the  financial  management 
community. 

with  release 
of5000.2R 

Barriers  - 
Risk  Aver¬ 
sion 

Fund 

PEO 

Make  risk  mitigation  funding  both  acceptable  and 
available. 

each  project 

Barriers  - 
Stakeholders 

Fund 

PM 

Program  budgets  must  include  funding  for  participa¬ 
tion  of  stakeholders  in  program  activities. 

Barriers  - 
Risk  Aver¬ 
sion 

IPT 

Senior 

Leadership 

Senior  leadership  should  demonstrate  support  for 
>  EA/SD  through  participation  in  spiral  development 

1  integrated  product  teams  (SDIPT). 

each  project 
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Work  Gp 

Class 

Agent 

Recommendation 

Due  date 

Barriers  - 
Risk  Aver¬ 
sion 

IPT 

DIPT 

Focus  program  oversight  on  the  program  manager’s  i 
risk  mitigation  plans. 

each  project 

Barriers  - 
Stakeholders 

IPT 

OSD 

Update  IPT  language  to  include  a  broader  definition 
of  “stakeholder”  for  spiral  development  projects. 

Barriers  - 
Stakeholders 

IPT 

PM 

The  number  of  IPTs  that  require  end-user  support 
must  be  limited  so  members  can  be  empowered  and 
knowledgeable. 

Barriers  - 
Stakeholders 

IPT 

PEO,  PM 

IPTs  should  be  given  the  authority  to  negotiate  re¬ 
quirements,  test  strategy,  and  budget  allocations. 

Barriers  - 
Education 

Study 

FFRDCs 

Develop  representative  risk  patterns  and  their  asso¬ 
ciated  mitigation  process  models  to  show  which  pat¬ 
terns  are  best  suited  to  EA/SD. 

Feb ’01 

Barriers  - 
Education 

Study 

Services 

Validate  EA/SD  models  through  industry  and  gov¬ 
ernment  pilot  projects  as  part  of  the  Simulation  Based 
Acquisition  Initiative. 

Sep ’01’ 

Barriers  - 
Education 

Study 

SIS  Office 

SEI  should  be  tasked  to  produce  a  case 
study/examples  document. 

Sep ’01 

Definition 

Study 

SIS  Office 

Document  experiential  case  studies 

Apr ’01 

Strategies 

Study 

C3I  and 
AT&L 

Develop  and  disseminate  detailed  case  studies  of  EA 
and  SD. 

Strategies 

Study 

Ferguson 

Identify  and  prototype  effective  ways  to  facilitate  the 
sustained  engagement  of  the  stakeholder  communi¬ 
ties. 

Mapping 

Study 

Etterwith 

SAEs 

Explore  EA  further  prior  to  imposing  it  as  a  policy. 
Evaluate  how  current  programs  would  have  fared 
under  EA.  Conduct  a  prototype  study. 

Barriers  - 
Education 

Train 

OSD 

Provide  a  contract  vehicle  for  E/VSD  facilitators  and 
mentors  to  support  early  adopters  of  EA/SD. 

Oct ’01 

Barriers  - 
Education 

Train 

OSD 

Require  that  EA/SD  curricula  be  incorporated  in  Pro¬ 
fessional  Military  Education  (PME)  and  Defense  Ac- 
quisitbn  University  (DAU)  for  stakeholders  as  well  as 
acquisition  professionals. 

Definition 

Train 

SIS  Office 

Use  definitions  to  develop  an  insert  that  can  be  put 
into  5000  Directives,  Regulations,  and  Instructions. 

Definition 

Train 

SIS  Office 

Develop  implementation  guidance  for  existing  publi¬ 
cations,  e.g.,  AFI 63-123. 

Apr ’01 

Definition 

Train 

SIS  Office 

Develop  guidebook  for  PMs  and  PEOs  addressing 
‘top  25”  implementation  questions. 

Apr ’01 

Definition 

Train 

DSMC 

Develop  education  and  training  modules  for  insertion 
into  /Acquisition  Management  courses. 

Aug  ’01 

Strategies 

Train 

SIS  Office 

Develop  a  cadre  of  industry  and  government  people 
knowledgeable  in  the  pragmatic  application  of  EA 
and  SD  who  are  readily  available  to  program  offices 
and  contractors. 

58 


CMU/SEI-2001-SR-005 


CMU/SE1-2001-SR-005 


59 


Acronyms 


This  list  omits  names  of  organizations  that  are  spelled  with  all  upper  case  letters. 
AC2ISRC  Aerospace  C2ISR  Command  (Air  Force) 

ACAT  Acquisition  Category 

ACTD  Advanced  Concept  Technology  Demonstration 

AF  Air  Force 

AFB  Air  Force  Base 

AFl  Air  Force  Instruction 

AFI  A  Air  Forces  Information  Agency 

/^POTEC  Air  Force  Operational  Test  and  Evaluation  Center 

ASC  Aeronautical  Systems  Center 

ASCON  Associate  Contractor 

Acquisition,  Technology,  and  Logistics  (a  Undersecretary  of  Defense) 

ATD  Advanced  Technology  Demonstration 

BES  Budget  Estimate  Submission 

C2ISR  Command  and  Control,  Intelligence,  Surveillance,  and  Reconnaissance 

Q4I  Command,  Control,  Communications,  Computers,  and  Intelligence 

CAOC  Combined  Air  Operations  Center 

CBO  Congressional  Budget  Office 

CeBASE  Center  for  Empirically  Based  Software  Engineering 

CECOM  US  Army  Communications-Electronics  Command 

CIO  Chief  Information  Officer 

cues  Chairman,  Joint  Chiefs  of  Staff 

CMM  Capability  Maturity  Model 

CMMI  Capability  Maturity  Model  Integration 

CMU  Carnegie  Mellon  University  (home  of  SEI) 

COTS  Commercial-Off-The-Shelf 
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CSE 

Center  for  Software  Engineering  (USC) 

CSM 

Center  for  Systems  Management 

DARPA 

Defense  Advanced  Research  Project  Administration 

DAU 

Defense  Acquisition  University 

DC 

District  of  Columbia 

DCMA 

Defense  Contract  Management  Agency 

DCMC 

Defense  Contract  Management  Command 

DD-21 

a  class  of  destroyers  for  the  2U*  century  (U.S.  Navy) 

DNA 

DeoxyriboNucleic  Acid 

DoD 

Department  of  Defense 

DoDI 

DoD  Instruction 

DSCOPS 

Deputy  Chief  of  Staff  for  Operations  and  Plans 

DSMC 

Defense  Systems  Management  College 

DUSD 

Deputy  Under  Secretary  of  Defense 

EA 

Evolutionary  Acquisition 

EA/SD 

Evolutionary  Acquisition  and  Spiral  Development 

EMD 

Engineering  and  Manufacturing  Development 

ESC 

Electronic  Systems  Conunand  (Air  Force) 

ESP 

Evolutionary  Spiral  Process  (Software  Productivity  Consortium) 

FAA 

Federal  Aviation  Agency 

FAQ 

Frequently  Asked  Questions 

FFRDC 

Federally  Funded  Research  and  Development  Center 

FSCV 

Future  Scout  Combat  Vehicle 

FY 

Fiscal  Year 

GAO 

Government  Accounting  Office 

GOTS 

Govemment-Off-The-Shelf 

lA 

Incremental  Acquisition 

IDA 

Institute  for  Defense  Analysis 

IDIQ 

Indefinite  Delivery  Indefinite  Quantity  (a  type  of  contract) 

IEEE 

Institute  for  Electrical  and  Electronic  Engineers 

INCOSE 

International  Council  on  Systems  Engineering 
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IOC 

Initial  Operating  Capability 

IPPD 

Integrated  Product  and  Process  Development 

IPT 

Integrated  Product  Team 

ISR 

Intelligence,  Surveillance,  and  Reconnaissance 

JAD 

Joint  Application  Development 

JEFX 

Joint  Expeditionary  Forces  Experiment 

JSF 

Joint  Strike  Fighter 

JWE 

Joint  Warfare  Exercise 

LCA 

Life  Cycle  Architecture 

LCO 

Life  Cycle  Objectives 

MAJCOM 

Major  Command  (USAF) 

MBASE 

Model-Based  Architecting  and  Software  Engineering 

NASA 

National  Aeronautical  and  Space  Administration 

NSF 

National  Science  Foundation 

OIPT 

Overarching  Integrated  Product  Team 

ORD 

Operational  Requirements  Document 

OSD 

Office  of  the  Secretary  of  Defense 

OUSD 

Office  of  the  Under  Secretary  of  Defense 

OUSD/AR 

OUSD  /  Acquisition  Reform 

OSD/PA&E 

OSD  /  Program  Analysis  and  Evaluation 

PEO 

Program  Element  Officer 

PM 

Program  Manager 

PME 

Professional  Military  Education 

PMI 

Program  Management  Institute 

PMO 

Program  Management  Office 

POM 

Program  Objectives  Memorandum 

PPBS 

Planning,  Programming,  and  Budgeting  System 

QFD 

Quality  Function  Deployment 

RDT&E 

Research,  Development,  Testing  and  Evaluation 

ROI 

Return  On  Investment 

RUP 

Rational  Unified  Process 
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SA-CMM 

Software  Acquisition  Capability  Maturity  Model 

SA/SD 

Structured  Analysis/Structured  Design 

SAE 

Service  Acquisition  Executive 

SAP 

Secretary  of  the  Air  Force 

SAF/AQ 

SAF/Acquisition 

SAF/AQI 

SAF/Acquisition:  Information  Dominance 

SAIC 

Science  Applications  International  Corporation 

SD 

Spiral  Development 

SDIPT 

Spiral  Development  Integrated  Product  Team 

SDM 

Spiral  Development  Model 

SEi 

Software  Engineering  Institute  (CMU) 

SEIR 

Software  Engineering  Information  Repository 

SIS 

Software  Intensive  Systems 

SMC 

Space  and  Missile  Systems  Center  (Air  Force) 

SPAWAR 

Space  and  Naval  Warfare  Systems  Command 

SPC 

Software  Productivity  Consortium 

SPO 

System  Program  Office 

SR 

Special  Report 

SW-CMM 

Software  Development  CMM 

TACOM 

Tactical  Command 

TMA 

Too  Many  Acronyms 

TSPR 

Total  System  Performance  Responsibility  (a  type  of  contract) 

UML 

Unified  Modeling  Language 

US 

United  States 

USAF 

United  States  Air  Force 

use 

University  of  Southern  California  (home  of  CSE) 

USD 

Under  Secretary  of  Defense 

USD/AT&L 

USD/ Acquisition,  Technology,  and  Logistics 

WG 

Work  Group  (of  the  workshop) 

WRAP 

Warfighter  Rapid  Acquisition  Process  (SAF/AQ) 
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